Isolating the Marginal Contribution of Distributed Context in LLM Attribute Inference

Isolating the Marginal Contribution of Distributed Context in LLM Attribute Inference Two recent studies establish that LLMs infer personal attributes from conversation history without explicit disclosure, and that they can do so early. Zaman and Garimella recover a user's age, gender, and country from donated ChatGPT logs at weighted F1 up to 0.90 after filtering out messages with explicit self-identification, finding that the median user becomes identifiable from just the first 5% of their history [1]. Ko et al.'s InferLink benchmark shows LLM agents combining scattered, individually non-identifying cues with public evidence to reconstruct real-world identities [2]. Neither paper isolates the mechanism behind the gain: whether additional context helps because it carries more relevant signal, or simply because the model has more text to reason over; two explanations that predict the same accuracy curve but demand different interventions. A system that flags risk by conversation length alone cannot distinguish a user who has shared many irrelevant messages from one whose messages are quietly converging on a confident profile. That is the gap this work targets: a dosage-controlled test of information content versus context volume, which neither prior paper runs. We propose a controlled, dosage-matched experiment. For synthetic user profiles with hidden attributes (geographic region, occupation, education level, language preference, living arrangement), we construct two length-matched conversation histories per profile: a random-content control, containing conversations unrelated to the target attribute, and a distributed-signal condition, containing conversations with weak, indirect clues relevant to the attribute; for instance, a passing mention of grading final exams in June as a clue toward occupation, or a comment about renewing a visa as a clue toward living arrangement; constrained so that no single message discloses the attribute directly. An explicit-disclosure condition serves as an upper bound. We evaluate inference accuracy, calibrated confidence, and abstention rate across progressively larger context sizes (1, 5, 10, 20, and 30 conversations), testing whether distributed-signal histories yield significantly higher inference performance than length-matched random histories. On profiles where inference is strong, we additionally perform fragment-level ablation, removing individual conversations one at a time to identify which specific turns disproportionately drive an attribute's inferability; a diagnostic not reported in either prior work. We hypothesize that (1) distributed-signal accumulation produces measurably higher inference accuracy than length-matched random accumulation, confirming the effect reflects information content rather than context volume alone, and (2) the marginal contribution of individual conversations is highly non-uniform, with a small number of fragments accounting for most of the inferential gain. A null result; no divergence between arms at matched length; would itself be informative, suggesting that “context accumulation” effects reported in prior work may partly reflect longer context windows rather than genuine signal aggregation. Pilot experiments across synthetic profiles and multiple model families are underway; we will report accuracy, calibration, and ablation results at the workshop. By quantifying the marginal, per-fragment contribution to inferential privacy leakage rather than only its aggregate presence, this work aims to inform interventions that can flag which accumulating interactions are actually driving risk, rather than treating conversation length alone as a proxy for exposure. References Zaman, S. M. Mehedi, and Kiran Garimella. “Inferential Privacy Leakage in Anonymized Conversational AI Logs.” arXiv:2605.23820 [cs], May 22, 2026. http://arxiv.org/abs/2605.23820. Ko, Myeongseob, Jihyun Jeong, Sumiran Singh Thakur, Gyuhak Kim, and Ruoxi Jia. “From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents.” arXiv:2603.18382 [cs, ai], March 19, 2026. http://arxiv.org/abs/2603.18382.