Riemannian Bilevel Optimization under the Polyak–Łojasiewicz Condition
Zhixuan Li ⋅ Yuyang Zhang ⋅ Luke Jones ⋅ Muztaba Syed ⋅ William Chang ⋅ Andi Han
Abstract
This paper studies bilevel optimization on Riemannian manifolds where the upper-level objective is nonconvex and the lower-level problem satisfies a Riemannian Polyak - Łojasiewicz condition rather than geodesic strong convexity. The classical hypergradient formula then breaks down, since the lower-level Hessian may be singular or indefinite away from the minimizer. We address this with an intrinsic regularized tangent-space formulation based on spectral clipping, and develop a Riemannian bilevel algorithm that avoids Hessian-inverse solves and inner-loop differentiation. We establish an $O(1/T)$ rate on the averaged squared Riemannian gradient mapping and $O(\varepsilon^{-1})$ iteration complexity, and extend the method to the stochastic finite-sum setting. Experiments on Stiefel, Grassmannian, and Poincar\'e-ball problems with rank-deficient Hessians (BCI~IV-2a, UCI Superconductivity) and Stiefel-constrained meta-learning on MiniImageNet show that the proposed method is the only one whose linear-system error stays at machine precision and whose Riemannian gradient norm decreases monotonically, while Hessian-inversion and Neumann-series baselines fail to descend; the stochastic complexity exhibits the predicted $\Theta(1/B)$ decay.
Chat is not available.
Successful Page Load