Beyond Sharpness: Two Local Stability Edges in Practical Muon
Zixu Wu ⋅ Hualin Zhang ⋅ Zhengquan luo ⋅ Zhiqiang Xu
Abstract
Practical Muon applies a finite Newton--Schulz map to a Nesterov proposal and carries momentum between steps. We analyze this real-arithmetic update and ask whether Hessian sharpness and the current weight step determine local stability. We show that the finite map derivative and stored momentum also matter. At an isotropic proposal where $g'(s)<0$, the derivative gives negative gain to relative singular value changes and positive gain to rotations. Positive definite curvature preserves both signs. In the joint update, negative modes reach the lower edge when a state multiplier equals $+1$, while positive modes reach the upper edge when one equals $-1$. At zero normalization offset, proposal rescaling preserves the transformed update and next weight but scales the derivative inversely. The same Hessian and next weight can therefore have different local stability. Controlled MLP and tiny Transformer tests verify both signs and the scale law. Across two checkpoints of one 124M run, all 96 saved matrix proposals have both derivative signs, and applying the saved map derivative to four restricted exact Hessians yields real modes beyond both edges.
Chat is not available.
Successful Page Load