From Inner Randomness to Outer Stability
Abstract
Two-phase optimizers such as local SGD and DiLoCo pass the displacement of K inner steps, the pseudo-gradient, to an outer step, which often uses Nesterov momentum. We study when the randomness of this pseudo-gradient is amplified by the outer momentum. For random centered quadratics, we derive an exact mean-square stability criterion in terms of the first and second moments of the block operator, the outer learning rate, and the outer momentum. In the scalar case, this criterion reduces to a closed-form necessary-and-sufficient threshold. We show that a mean-response analysis declares the training trajectory stable in regimes where the second moment of the error grows without bound, and can reverse the stability ranking of different numbers of inner steps. We then apply our analyses to small language models using a paired perturbation probe that estimates these moments at checkpoints and uses them to predict the stability boundary. Running the outer optimizer from the same checkpoints, we find that the transitions in the across-seed variance are generally consistent with the predicted boundary.