Not Your Diary, Just Your Topic: Grading the Wrong-User Control
Abstract
A memory module that scores well has not thereby been shown to read the right user's memory. We audit that gap on a frozen decoder whose user history is compressed into eight vectors, and report what the audit finds, including where it overturns our own earlier readings. The control the field runs, swapping in a randomly drawn other user, answers a coarser question than it is usually read as answering, because it moves the subject matter and the person at once. Grading the swap by how similar the replacement's profile is turns the control into a curve: the matched history wins by 10.1 points of accuracy against a randomly drawn user, by 3.6 against the most similar one the split contains, and by 1.2 against a profile assembled to match the user's subject matter. User-specific conditioning is real, and across four ways of building the negative it lands between 0.6 and 4.0 points, well under the 10.1 the standard control reports. It is also narrow: of five LaMP tasks, four leave the conditions on top of each other, so what we offer is a way to measure the decomposition rather than a figure for how large it usually is. Where the memory enters matters more than how many parameters enter with it: gated cross-attention leads a soft prefix over the same history encoder by 5.5 points, of which about one is output format, while growing the prefix makes it monotonically worse, by 7.0 points across 1.76x the parameters, with training loss rising and output validity flat. Giving the prefix the injected model's gated zero-initialized branch does not close the gap. Three things we previously reported do not survive their own controls: the per-user concentration of the memory effect is what a permutation null with matched item counts produces, the flat capacity curve above one injected layer does not reproduce on an audited harness, and the scale contrast we drew from a smaller decoder came from training it on a shorter schedule.