Persistent Data-Order Effects Precede Directional Alignment in Language Models

Data order changes optimization trajectories, but endpoint divergence alone does not reveal how the corresponding trace develops or whether it is recoverable. We measure output divergence and raw parameter-space endpoint-direction alignment along public Pythia-160M/PolyPythias trajectories trained on 300 billion tokens. Three correlated order-only pairs reach symmetric KL of 0.290–0.293 nats, compared with 0.306 nats for one joint initialization-and-order pair. The central finding is a temporal dissociation. At step 16,000, the order/joint divergence ratio is already 0.932 for one pair, while its raw endpoint-direction cosine is only 0.294. These are distinct readouts, not a causal decomposition. Most parameter-block grafts then leave a predeclared language-modeling loss band, and none of the tested Fisher estimators predicts all three probe directions within 25% relative error. The comparisons use one training-distribution shard and lack an initialization-only control. Persistent order dependence is measurable. Functional recoverability and the cost of rewriting completed optimization histories remain open.