Training Dynamics as Local Energy Transport on Computational Graphs
Donghee Lee ⋅ Hye-Sung Lee ⋅ Seungyong Lee ⋅ Jaeok Yi
Abstract
Continuous-time descriptions of optimization usually treat neural-network training as motion in parameter space, leaving the computational graph implicit. Using the existing bulk--boundary decomposition, we evaluate local energy balance residuals and synaptic energy-current patterns along discrete momentum training. For a fully connected $\tanh$ network on XOR, the reported residuals, computed with the step-dependent damping convention stated below, have full-batch median per-step values of order $3\times10^{-6}$ and aggregate relative values below $10^{-3}$ in both full-batch and single-sample stochastic training. A step-size scan at one initialization exhibits approximately linear residual scaling over the tested range, although its direct interpretation as continuum consistency requires matching the damping normalization. Full-batch currents are almost always positive in the input-to-output orientation, while stochastic updates show frequent reversals. Nevertheless, absolute-current shares across the three measured bulk interfaces are similar in both regimes and stabilize after an early transient, while current magnitudes continue to decay. These findings characterize the observed trajectories under the chosen BBD energy convention and within the studied architecture and task.
Chat is not available.
Successful Page Load