Are Self-Evolving Agents Continual Learners?

Large language model agents are often deployed in environments where new knowledge, tools, and tasks keep arriving. Such an agent is therefore expected to adapt continually, learning from the tasks it encounters without forgetting earlier ones. This resembles the continual learning problem, but at a pace that conventional gradient updates cannot sustain. Nonparametric self-evolving agents are a natural alternative: they adapt by rewriting their own harness, such as prompts, memory, and skills, from task feedback while the model weights stay frozen. In this work, we investigate whether current nonparametric self-evolving agents are effective continual learners by constructing task streams from three agentic benchmarks, ScienceWorld, BFCL, and CodeContests+, in which the agent is trained on each of them consecutively. We evaluate three representative self-evolution methods, ReasoningBank, GEPA, and Meta-Harness, and find that they improve average performance in eight of the nine combinations of method and stream, and that a benchmark trained at the end of a stream can surpass the same benchmark trained alone. Forward and backward transfer, however, expose two failures that the average conceals. Forward transfer is often negative and arises from interference between benchmarks, so a benchmark can be served below the initial agent until its own tasks arrive. Self-modifying methods forget by overwriting, such that one harness edit can erase a benchmark's competence for the rest of the stream. We conclude that current self-evolving agents are incomplete continual learners, and call for methods that retain positive transfer while excluding interference and that update without overwriting.