Graph or Sequence? Understanding Structural Explanations in Chemical Language Models

Chemical language models (cLMs) process molecular graphs through SMILES strings, making mechanistic interpretation difficult when molecular structure correlates with properties of its textual representation. We study this ambiguity using structural attention. Re-serialising the same molecule while tracking the same atoms weakens localisation as their token span grows, while connected atom sets remain more localised than disconnected sets after matching molecule identity, element composition, and span. The connectivity advantage is largest when relevant tokens are dispersed, indicating that both sequence locality and connectivity-associated information shape routing. We then test how stable the resulting mechanistic interpretation is. Fixed-head transfer changes substantially with the localisation metric; restoring the same components recovers 93-99\% of intact behaviour under zero ablation but only 9-20\% under resample ablation; similar top-1 recovery can hide a $5.2\times$ difference in output-distribution divergence; and the largest measured pathways originate from MLPs rather than attention heads. Finally, an apparent frozen-representation advantage over classical molecular features disappears when the classical baseline receives a stronger learner. These results show that reliable mechanistic claims require controls for the representation, explanation metric, intervention, and evaluation baseline.