Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell
Hochan Son ⋅ Kyungdoe Han ⋅ Jaehan Koh ⋅ Xiaowu Dai ⋅ Wenlu Xu ⋅ Guang Cheng
Abstract
Decomposing long-context inference across cooperating agents bounds the \emph{active} KV cache per call rather than total evidence, which matters when KV-cache memory binds. Many such systems add a persistent tier storing and recalling reasoning traces, usually validated by an ablation reporting an accuracy gain. We measure both on one three-tier agent architecture. Decomposition delivers: peak KV working set of $14.3$MiB per query against $35.5$ and $35.3$MiB for single-pass and retrieval-augmented baselines. The persistent tier does not: across eight controlled dataset pairs at $n{=}100$ per arm it costs $+0.368$MiB$[+0.167,+0.590]$ of peak cache and produces no detectable accuracy change ($+0.015$, 95\% CI $[-0.011,+0.046]$). We argue the null is \emph{structural}: single-question benchmarks supply each item with its own evidence and score it independently, and correctness requires resetting stored traces between conditions, so recall has nothing informative to retrieve. Reaching it took four measurement corrections---three inflating the apparent benefit, the fourth making an effect that size look resolvable---none visible in the results table. We give the conditions an agent-memory ablation must satisfy and detection procedures that need no knowledge of the specific defect.
Chat is not available.
Successful Page Load