The Echo Chamber of One: Measuring Memory-Conditioned Narrowing in Personal AI
Abstract
Long-term memory can improve continuity in personal AI, but recall-centric evaluation leaves a complementary failure unpenalized: historical user signals can become a self-reinforcing prior that narrows the responses, proposals, and identities made salient to the user. Prior work has warned that persistent personal AI may produce self-echo chambers; we contribute a memory-specific operationalization of that risk. First, a dual-collapse model treats memory failure as two independent risks -- assimilation to the user's past and alienation through forgetting -- and requires both to be measured in one longitudinal run, since one-sided evaluation leaves drift toward the unmeasured failure unpenalized. Second, Counterfactual Memory Ablation (CMA) pairs each scripted trajectory with a frozen no-memory condition, identifying the causal effect of the memory condition on agent outputs and separating grounded expansion from anchoring. Third, developmental forgetting recasts a user's own request to outgrow a past self as reweighting rather than deletion, measured by Graduation Leakage Rate. As a case study, we stress-test governance rules over 50 deterministic 36-month synthetic lives and a four-policy user panel: ablation directions hold in at least 191 of 200 paired runs, while magnitudes remain consequences of the specified synthetic policies -- executable model-checking, not estimates of human behavior. We position the framework against recent work on over-personalization and memory-induced sycophancy, and identify validated user simulation as the gate between system-side effects and claims about users.