Mode Dynamics: From Neural Network Parameter Trajectories to Evolving Function-Space Effects

We introduce Mode Dynamics, a framework for trajectory-level mechanistic analysis that connects neural network parameter trajectories to their evolving effects in function space. From observed parameter updates, we extract fixed orthogonal trajectory modes, express each update through time-varying amplitudes, and map each mode to its local effect on model outputs. This separates compact modal structure in the realized update trajectory from the nonlinear, state-dependent parameter-to-function map. In controlled regression, compact mode prefixes reconstruct both local output changes and the learned function globally. Accurate global reconstruction requires evaluating induced effects along the training path; freezing them at one reference state fails despite preserving the same modes and amplitudes. Across 675 decoder-only transformer runs, modal complexity, measured by participation ratio, varies systematically with task and data complexity, while compact prefixes reconstruct logits and decisions. Together, these results provide a reconstruction-tested mechanistic account of path-dependent behavioral change. The observed modal concentration is an empirical finding in these settings, not a low-dimensionality assumption of the framework.