Stable Stationary Orders, Unstable Fresh Reshuffling: An Exact Heavy-Ball Counterexample
Abstract
Does stability for every fixed data order guarantee stability under fresh reshuffling? For conventional constant-step heavy ball, it does not. We construct an exact counterexample using two scalar positive quadratics with a common minimizer. On an explicit open step-size interval, the component maps, the average-curvature map, and both repeated-permutation schedules are strictly Schur stable. Yet choosing a fresh permutation independently and uniformly at each epoch makes the expected epoch-boundary state grow exponentially from every nonzero deterministic initialization. At a rational center, an exact uncentered second-moment calculation also proves expected-objective divergence for every such initialization. A certified positive Lyapunov bound gives almost-sure exponential divergence on a narrow surrounding band. The expected-state separation persists on an explicit open set of noncommuting positive-definite two-dimensional Hessian pairs. The underlying switching and moment mechanisms are classical. Our contribution is their exact realization and certification in this optimizer. Randomness is not necessary: alternating the two epoch maps is unstable too. The example therefore separates stationary orders from fresh reshuffling, not arbitrary switching from random switching.