MemForge: Auditable Self-Evolution of Agent Memory Systems via Structured Coordinate Descent and Generational Review

Long-term memory is essential for large language model (LLM) agents: information streams that exceed the context window must be persisted, organized, and retrieved on demand. Yet memory design has no converged answer, and hand-crafted designs traverse a vast space with sparse rewards. Letting agents evolve their own memory systems is a natural alternative, but free-form self-evolution suffers from three systematic failure modes: overfitting to the few observed samples, convergent collapse onto stereotyped designs with irreversible information loss, and exploration that produces complexity rather than gains; anti-overfitting instructions, the intuitive mitigation, neither stabilize nor transfer. We introduce MemForge, which recasts memory self-evolution as controlled search in a structured design space: (i) an atomic optimization space of seven orthogonal operator dimensions; (ii) a coordinate descent protocol that validates every modification and forces a per-dimension ablation log, turning exploration time into recomputable evidence; (iii) a Worker-Reviewer generational mechanism whose outer reviewer holds hidden test scores to verify attribution, audit overfitting, and seed the next generation with type-level directions and the full code lineage. On LoCoMo and MemoryAgent-Bench, against six strong baselines including ALMA and MemEvolve, MemForge achieves the best overall scores on both benchmarks under both backbones: LoCoMo F1 0.603 and 0.612 against the best baseline 0.563 and 0.578, and MAB accuracy 0.765 and 0.797 against 0.760 and 0.790, also leading the Temporal and Single-Hop LoCoMo subsets and the EventQA, ICL, and DetectiveQA families under both backbones. The results highlight the value of defining the search space: seven orthogonal dimensions turn memory self-evolution into an auditable search that yields designs which transfer across benchmarks and backbones.