Memorize Fast and Slow: Symbolic Memory Revision for Test-time Conversational Agents

Conversational agents must revise memory when users correct, qualify, or retract earlier preferences or facts, while preserving the rest of their memory state. Using a neurosymbolic lens, we address an agent's symbolic scaffold, the explicit structure surrounding a neural model that organizes the evidence available to it, maintains persistent knowledge, and governs how that knowledge is revised and retrieved. Drawing on Complementary Learning Systems theory, we claim this scaffold pairs a fast memory of recorded events and temporary claims with a slow memory of integrated, revisable knowledge. We consider fast memory as the context and an immutable event log, and slow memory as a knowledge graph reconstructed through replay. At write time, Python validation and Scallop consistency rules gate which candidate facts enter the graph; at query time, hybrid graph and dense retrieval return a bounded selection of active facts to the model's context. We evaluate our approach on a synthetic benchmark of multi-user conversations with 120 matched conditions per surface. Pooled across both surfaces, our system achieves 67.50% exact match at a 4K context budget, compared with 40.42% for Graphiti, 60.83% for structured memory, and 30.42% for sliding context windows. These results suggest that explicitly governed memory revision can improve personalized agents, especially when relevant evidence is distant. Our code is available at https://github.com/ArS377/Memorize-Fast-and-Slow.