What the Agent May Say Beats What the Agent Sees: The Grounding Vocabulary Dominates the Input Channel in Web Agents

Multimodal web agents are handed every channel at once, a screenshot alongside the accessibility tree, and charged for each at every step: one task here costs 143k input tokens before the agent has said anything. We hold the agent constant (one action set, one step budget, one memory, one prompt) and vary only its interface, over 75 tasks on 8 deterministic replicas of real websites and two vision-language models (claude-sonnet-4.6, qwen3-vl-235b): 1,992 scored episodes. One axis removes input channels; a second, independent axis changes the grounding vocabulary, whether the agent names its target by the identifier the accessibility tree assigns or by pixel coordinates. The vocabulary dominates the observation. Swapping identifiers for coordinates costs $0.147$ (sonnet, McNemar $p=0.007$) and $0.280$ (qwen, $p<0.0001$), the only contrasts in the matrix to clear our resolution threshold, while deleting the entire accessibility tree costs $0.080$ and $0.040$. Changing how the agent may answer is thus 1.8 to 7 times more damaging than removing its largest input channel. The vocabulary is nearly free in tokens, and the most token-efficient interface is not the full-observation one. Degradation also differs in kind: under identical degraded grounding one model falls silent and the other invents, claiming a task done when it was not at $0.118$ for sonnet against $0.808$ for qwen. Success rate scores both alike, so an interface cannot be chosen on success rate alone.