Timezone: »

 
Poster
Stochastic Halpern Iteration with Variance Reduction for Stochastic Monotone Inclusions
Xufeng Cai · Chaobing Song · Cristóbal Guzmán · Jelena Diakonikolas

Thu Dec 01 02:00 PM -- 04:00 PM (PST) @ Hall J #818
in Poster Session 6 »
We study stochastic monotone inclusion problems, which widely appear in machine learning applications, including robust regression and adversarial learning. We propose novel variants of stochastic Halpern iteration with recursive variance reduction. In the cocoercive---and more generally Lipschitz-monotone---setup, our algorithm attains $\epsilon$ norm of the operator with $\mathcal{O}(\frac{1}{\epsilon^3})$ stochastic operator evaluations, which significantly improves over state of the art $\mathcal{O}(\frac{1}{\epsilon^4})$ stochastic operator evaluations required for existing monotone inclusion solvers applied to the same problem classes. We further show how to couple one of the proposed variants of stochastic Halpern iteration with a scheduled restart scheme to solve stochastic monotone inclusion problems with ${\mathcal{O}}(\frac{\log(1/\epsilon)}{\epsilon^2})$ stochastic operator evaluations under additional sharpness or strong monotonicity assumptions. Finally, we argue via reductions between different problem classes that our stochastic oracle complexity bounds are tight up to logarithmic factors in terms of their $\epsilon$-dependence.
Keywords: stochastic · monotone inclusion · variance reduction · Halpern iteration · min-max optimization · last iterate convergence

Author Information

Xufeng Cai (University of Wisconsin-Madison)
Chaobing Song (University of Wisconsin-Madison)
Cristóbal Guzmán (PUC-Chile)
Jelena Diakonikolas (University of Wisconsin-Madison)

More from the Same Authors