Critical-Point Geometry and Saddle Escape in Overparameterized Neural Optimization

Stationary-point analysis is subtle in overparameterized neural optimization because interpolating solutions are typically non-isolated. We give a critical-point account that separates benign interpolation degeneracy from strict saddle structure. For analytic two-layer networks with square loss, any full-rank interpolating minimum is Morse-Bott: the Hessian kernel is exactly the tangent space of the interpolation manifold, and the normal directions are positive. Adding a small ridge term and a generic linear tilt yields a proper Morse objective, so classical Morse inequalities imply that $c_0$ minima force at least $c_0-1$ index-1 saddles. We also prove a local gradient-flow and gradient-descent saddle-escape bound with logarithmic dependence on perturbation size and inverse dependence on negative curvature. Small full-Hessian experiments validate the critical-point geometry, generic-tilt non-degeneracy, and predicted escape-time scaling.