From Inner Randomness to Outer Stability

We study when randomness in a K-step pseudo-gradient is amplified by the outer Nesterov momentum of two-phase optimizers. For random centered quadratics, we derive an exact mean-square stability criterion in terms of the first and second moments of the block operator, the outer learning rate, and the outer momentum. In the scalar case, this criterion reduces to a closed-form necessary-and-sufficient threshold. We show that a mean-response analysis declares the training trajectory stable in regimes where the second moment of the error grows without bound, and can reverse the stability ranking of inner block lengths. We then apply our analyses to small language models using a paired perturbation probe that estimates these moments at checkpoints and uses them to predict the stability boundary. Running direct rollouts from the same checkpoint, we find that the observed variance transitions are generally consistent with the predicted boundary.