Timezone: »
Invited speaker: Concentration for matrix products, and convergence of Oja’s algorithm for streaming PCA, Rachel Ward
Rachel Ward
We present new nonasymptotic growth and concentration bounds for a product of independent random matrices, similar in spirit to concentration for sums of independent random matrices developed in the previous decade. Our matrix product concentration bounds provide a new, direct convergence proof of Oja's algorithm for streaming Principal Component Analysis, and should be useful more broadly for analyzing the convergence of stochastic gradient descent for certain classes of nonconvex optimization problems, including neural networks. This talk covers joint work with Amelia Henriksen, De Huang, Jon Niles-Weed, and Joel Tropp.
Author Information
Rachel Ward (UT Austin)
More from the Same Authors
-
2021 Spotlight: Bootstrapping the Error of Oja's Algorithm »
Robert Lunde · Purnamrita Sarkar · Rachel Ward -
2021 Poster: Bootstrapping the Error of Oja's Algorithm »
Robert Lunde · Purnamrita Sarkar · Rachel Ward -
2020 : Rachel Ward »
Rachel Ward -
2020 Poster: Implicit Regularization and Convergence for Weight Normalization »
Xiaoxia Wu · Edgar Dobriban · Tongzheng Ren · Shanshan Wu · Zhiyuan Li · Suriya Gunasekar · Rachel Ward · Qiang Liu