Poster
Finding Second-Order Stationary Points in Nonconvex-Strongly-Concave Minimax Optimization
Luo Luo · Yujun Li · Cheng Chen
Keywords: [ cubic regularization ] [ minimax optimization ] [ second-order optimization ]
Abstract:
We study the smooth minimax optimization problem minxmaxyf(x,y), where f is ℓ-smooth, strongly-concave in y but possibly nonconvex in x. Most of existing works focus on finding the first-order stationary point of the function f(x,y) or its primal function P(x)≜maxyf(x,y), but few of them focus on achieving the second-order stationary point, which is essential to nonconvex problems. In this paper, we propose a novel approach for minimax optimization, called Minimax Cubic Newton (MCN), which could find an O(ε,κ1.5√ρε)-second-order stationary point of P(x) with calling O(κ1.5√ρε−1.5) times of second-order oracles and ˜O(κ2√ρε−1.5) times of first-order oracles, where κ is the condition number and ρ is the Lipschitz continuous constant for the Hessian of f(x,y). In addition, we propose an inexact variant of MCN for high-dimensional problems to avoid calling the expensive second-order oracles. Instead, our method solves the cubic sub-problem inexactly via gradient descent and matrix Chebyshev expansion. This strategy still obtains the desired approximate second-order stationary point with high probability but only requires ˜O(κ1.5ℓε−2) Hessian-vector oracle calls and ˜O(κ2√ρε−1.5) first-order oracle calls. To the best of our knowledge, this is the first work that considers the non-asymptotic convergence behavior of finding second-order stationary points for minimax problems without the convex-concave assumptions.
Chat is not available.