The Many Faces of One Head: Rethinking Attention Diversity in Recommenders

Attention weights in recommender systems are widely used as post-hoc explanations of what the model attended to when making a recommendation. This practice inherits, without stating so, an assumption from language-model interpretability: that individual attention heads implement distinguishable functions. If they do not, a single-head heatmap shows one function the layer runs several times, not evidence about the particular recommendation. We report a work in progress that tests this assumption in two attention-based recommender families; sequential prediction (SASRec) and collaborative filtering over a user-item graph (GAT-Rec); on 5 amazon review datasets. We port three head-level probes from mechanistic interpretability (direct logit attribution, causal activation patching, zero-ablation) and add two measurements required to interpret the probes' output in this setting: pairwise cosine distance between heads' attention distributions, and a specialisation readout of attention mass over interpretable proxies (recency, repeat, popularity, degree). We further run a head-count sweep at $H \in \{2, 4, 8, 16\}$ on a dataset for both models. Across the five datasets, the three head-level probes routinely disagree on which head is dominant, per-head attention distributions are near-identical (mean pairwise distance $0.015$ for SASRec, $0.0002$ for GAT-Rec), and adding heads does not induce specialisation. Under this regime, the head-level ranking questions the probes were built to answer; are not well-posed, and a single-head attention map; used as an explanation summarises one function drawn many times.