Timezone: »

 
Poster
Accelerated Stochastic Matrix Inversion: General Theory and Speeding up BFGS Rules for Faster Second-Order Optimization
Robert Gower · Filip Hanzely · Peter Richtarik · Sebastian Stich

Wed Dec 05 02:00 PM -- 04:00 PM (PST) @ Room 210 #41
in Wed Poster Session B »

We present the first accelerated randomized algorithm for solving linear systems in Euclidean spaces. One essential problem of this type is the matrix inversion problem. In particular, our algorithm can be specialized to invert positive definite matrices in such a way that all iterates (approximate solutions) generated by the algorithm are positive definite matrices themselves. This opens the way for many applications in the field of optimization and machine learning. As an application of our general theory, we develop the first accelerated (deterministic and stochastic) quasi-Newton updates. Our updates lead to provably more aggressive approximations of the inverse Hessian, and lead to speed-ups over classical non-accelerated rules in numerical experiments. Experiments with empirical risk minimization show that our rules can accelerate training of machine learning models.

Keywords: Convex Optimization
[ Paper]  [ Poster] 

Author Information

Robert Gower (ParisTech)
Filip Hanzely (KAUST)

Research Assistant Professor, optimization/machine learning

Peter Richtarik (KAUST)
Sebastian Stich (EPFL)

Dr. [Sebastian U. Stich](https://sstich.ch/) is a faculty at the CISPA Helmholtz Center for Information Security. Research interests: - *methods for machine learning and statistics*—at the interface of theory and practice - *collaborative learning* (distributed, federated and decentralized methods) - *optimization for machine learning* (adaptive stochastic methods and generalization performance)

More from the Same Authors