Augmented Lagrangian for Constrained Learning

While Lagrangian duality has become a popular tool for addressing constrained learning problems, augmented Lagrangian methods remain largely underexplored. These methods mitigate the duality gap in non-convex settings while requiring only minimal modifications to implement. We establish strong duality under mild conditions, prove convergence of dual ascent algorithms to feasible and optimal primal solutions, and provide PAC-style generalization guarantees. Finally, we demonstrate its effectiveness on a fairness-constrained classification task.