Same Fact, Different Channel: Bounding User–Tool Role Effects in Long-Context Retrieval
Abstract
Long-horizon agents accumulate evidence through heterogeneous conversational channels, including user turns, tool returns, and prior assistant output. Because chat models are post-trained to distinguish these channels for authority and safety, attributing the same fact to a different role may also change how accessible it is at inference time. We isolate this effect using 12,000 paired long-context prompts that are identical after normalizing the role marker, with a verified one-token difference between arms, spanning two Llama-3.2 model sizes, five context lengths from 4K to 128K, and five evidence depths. Using teacher-forced gold log-probability, we test equivalence against a prespecified margin of ±0.20 nats. All ten model-by-length cells satisfy equivalence; the pooled USER−TOOL difference is −0.0028 nats (95% CI [−0.0104, +0.0047]). Across six matched model-by-length cells through 32K, the cell estimates reproduce on 240 held-out seeds (same sign in 6/6 cells, Pearson r = 0.989). Greedy decoding on confirmatory items gives a paired accuracy difference of +0.0003 (95% CI [−0.0056, +0.0061]), and a post-confirmatory contrast using full tool-call traces remains within the same log-probability bound at 4K and 16K. The measurement is sensitive to structural perturbation: replacing the role label with a length-matched nonce shifts gold log-probability by +0.075 to +0.091 nats over 4K–32K, whereas USER−TOOL effects vary in sign and cancel in aggregate. The aggregate bound is not uniform over evidence position: 7 of 50 depth-resolved cells fail equivalence, including three wholly beyond the margin, and pooled effects vary from +0.086 nats at depth 0.1 to −0.056 at 0.7 before partially recovering at 0.9. For these models and this controlled retrieval setting, channel attribution produces no consistent aggregate accessibility penalty, while moving the same evidence to a different depth within one context length changes gold probability by 0.061 to 0.669.