Can Language Models Infer Changed Preferences in Long-Horizon Personalization?
Zarreen Reza
Abstract
Benchmarks for long-horizon personalization ask whether a model tracks a user preference after a life event silently changes it. HorizonBench (S. S. Li, et al. 2026) diagnoses failure with the $\textit{stale-option rate}$: how often a wrong answer is the user's original, now-outdated preference. A high rate is read as $\textit{anchoring}$ on the old value, separating belief-update failure from retrieval failure. This presupposes an earlier step that is never measured: $\textit{event-to-preference inference}$, deriving the new value from the event that caused it. We add a positive control for that step. On Qwen2.5-7B, recall of the current preference given the event episode is at chance (20.1\%, indistinguishable from the 19.9\% no-context baseline), while stating it outright lifts recall to 46.6\%. Under this task formulation the models do not derive the updated preference from the event, so the downstream stale-option rate cannot be read as belief-update failure. We report initial behavioral findings and the mechanistic follow-up to explain such behaviors.
Chat is not available.
Successful Page Load