Self-Evolving Agents Should Build Internal and External Models of the World
Abstract
Self-evolving agents that learn continuously from streams of information are an emerging frontier in artificial intelligence. This capability is critical for efficiently updating computationally expensive systems such as foundation models. When agents explore their environment and interact with other agents across multiple tasks, two sources of non-stationarity arise: shifting task distributions and co-evolving peers. However, frameworks for continual multi-agent reinforcement learning (CMARL) that support effective policy improvement remain scarce. In this position paper, we argue that training self-evolving agents should incorporate two classes of model-based principles: internal models that allow an agent to compare policy checkpoints, monitor its learning capacity, and represent its current objective; and external models that learn environment dynamics and the behavior of other agents. We anticipate that discussing these principles will guide the design of algorithms, architectures, and benchmarks that explicitly evaluate model-based components in CMARL.