Workshop
OPT2020: Optimization for Machine Learning
Courtney Paquette 路 Mark Schmidt 路 Sebastian Stich 路 Quanquan Gu 路 Martin Takac
Fri 11 Dec, 3:15 a.m. PST
Optimization lies at the heart of many machine learning algorithms and enjoys great interest in our community. Indeed, this intimate relation of optimization with ML is the key motivation for the OPT series of workshops.
Looking back over the past decade, a strong trend is apparent: The intersection of OPT and ML has grown to the point that now cutting-edge advances in optimization often arise from the ML community. The distinctive feature of optimization within ML is its departure from textbook approaches, in particular, its focus on a different set of goals driven by "big-data, nonconvexity, and high-dimensions," where both theory and implementation are crucial.
We wish to use OPT 2020 as a platform to foster discussion, discovery, and dissemination of the state-of-the-art in optimization as relevant to machine learning. And well beyond that: as a platform to identify new directions and challenges that will drive future research, and continue to build the OPT+ML joint research community.
Invited Speakers
Volkan Cevher (EPFL)
Michael Friedlander (UBC)
Donald Goldfarb (Columbia)
Andreas Krause (ETH, Zurich)
Suvrit Sra (MIT)
Rachel Ward (UT Austin)
Ashia Wilson (MSR)
Tong Zhang (HKUST)
Instructions
Please join us in gather.town for all breaks and poster sessions (Click "Open Link" on any break or poster session).
To see all submitted paper and posters, go to the "opt-ml website" at the top of the page.
Use RocketChat or Zoom link (top of page) if you want to ask the speaker a direct question during the Live Q&A and Contributed Talks.
Schedule
Fri 3:15 a.m. - 3:50 a.m.
|
Welcome event (gather.town) ( Social event/Break ) > link | Quanquan Gu 路 Courtney Paquette 路 Mark Schmidt 路 Sebastian Stich 路 Martin Takac 馃敆 |
Fri 3:50 a.m. - 4:00 a.m.
|
Welcome remarks to Session 1
(
Opening remarks
)
>
|
Sebastian Stich 馃敆 |
Fri 4:00 a.m. - 4:20 a.m.
|
Invited speaker: The Convexity of Learning Infinite-width Deep Neural Networks, Tong Zhang
(
Talk
)
>
SlidesLive Video |
Tong Zhang 馃敆 |
Fri 4:20 a.m. - 4:30 a.m.
|
Live Q&A with Tong Zhang (Zoom)
(
Q&A
)
>
|
Sebastian Stich 馃敆 |
Fri 4:30 a.m. - 4:50 a.m.
|
Invited speaker: Adaptation and universality in first-order methods, Volkan Cevher
(
Talk
)
>
|
Volkan Cevher 馃敆 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed talks in Session 1 (Zoom) ( Multiple talks ) > link | Sebastian Stich 路 Laurent Condat 路 Zhize Li 路 Ohad Shamir 路 Tiffany Vlaar 路 Mohammadi Zaki 馃敆 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed Video: Constraint-Based Regularization of Neural Networks, Tiffany Vlaar
(
Talk
)
>
link
SlidesLive Video |
Tiffany Vlaar 馃敆 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed Video: Can We Find Near-Approximately-Stationary Points of Nonsmooth Nonconvex Functions?, Ohad Shamir ( Talk ) > link | Ohad Shamir 馃敆 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed Video: Employing No Regret Learners for Pure Exploration in Linear Bandits, Mohammadi Zaki
(
Talk
)
>
link
SlidesLive Video |
Mohammadi Zaki 馃敆 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed Video: Distributed Proximal Splitting Algorithms with Rates and Acceleration, Laurent Condat
(
Talk
)
>
link
SlidesLive Video |
Laurent Condat 馃敆 |
Fri 5:00 a.m. - 5:30 a.m.
|
Contributed Video: PAGE: A Simple and Optimal Probabilistic Gradient Estimator for Nonconvex Optimization, Zhize Li
(
Talk
)
>
link
SlidesLive Video |
Zhize Li 馃敆 |
Fri 6:00 a.m. - 6:50 a.m.
|
Poster Session 1 (gather.town) ( Poster session ) > link |
26 presentersLaurent Condat 路 Tiffany Vlaar 路 Ohad Shamir 路 Mohammadi Zaki 路 Zhize Li 路 Guan-Horng Liu 路 Samuel Horv谩th 路 Mher Safaryan 路 Yoni Choukroun 路 Kumar Shridhar 路 Nabil Kahale 路 Jikai Jin 路 Pratik Kumar Jawanpuria 路 Gaurav Kumar Yadav 路 Kazuki Koyama 路 Junyoung Kim 路 Xiao Li 路 Saugata Purkayastha 路 Adil Salim 路 Dighanchal Banerjee 路 Peter Richtarik 路 Lakshman Mahto 路 Tian Ye 路 Bamdev Mishra 路 Huikang Liu 路 Jiajie Zhu |
Fri 6:50 a.m. - 7:00 a.m.
|
Welcome remarks to Session 2
(
Opening remarks
)
>
|
Martin Takac 馃敆 |
Fri 7:00 a.m. - 7:20 a.m.
|
Invited speaker: Adaptive Sampling for Stochastic Risk-Averse Learning, Andreas Krause
(
Talk
)
>
SlidesLive Video |
Andreas Krause 馃敆 |
Fri 7:20 a.m. - 7:30 a.m.
|
Live Q&A with Andreas Krause (Zoom)
(
Q&A
)
>
|
Martin Takac 馃敆 |
Fri 7:30 a.m. - 7:50 a.m.
|
Invited speaker: Practical Kronecker-factored BFGS and L-BFGS methods for training deep neural networks, Donald Goldfarb
(
Talk
)
>
SlidesLive Video |
Donald Goldfarb 馃敆 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed talks in Session 2 (Zoom)
(
Multiple talks
)
>
|
Martin Takac 路 Samuel Horv谩th 路 Guan-Horng Liu 路 Nicolas Loizou 路 Sharan Vaswani 馃敆 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed Video: Adaptivity of Stochastic Gradient Methods for Nonconvex Optimization, Samuel Horvath
(
Talk
)
>
link
SlidesLive Video |
Samuel Horv谩th 馃敆 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed Video: Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence, Nicolas Loizou
(
Talk
)
>
link
SlidesLive Video |
Nicolas Loizou 馃敆 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed Video: DDPNOpt: Differential Dynamic Programming Neural Optimizer, Guan-Horng Liu
(
Talk
)
>
link
SlidesLive Video |
Guan-Horng Liu 馃敆 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed Video: Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search), Sharan Vaswani
(
Talk
)
>
link
SlidesLive Video |
Sharan Vaswani 馃敆 |
Fri 8:00 a.m. - 8:30 a.m.
|
Contributed Video: How to make your optimizer generalize better, Sharan Vaswani
(
Talk
)
>
link
SlidesLive Video |
Sharan Vaswani 馃敆 |
Fri 8:30 a.m. - 9:00 a.m.
|
Break (gather.town) link | 馃敆 |
Fri 9:00 a.m. - 9:20 a.m.
|
Invited speaker: SGD without replacement: optimal rate analysis and more, Suvrit Sra
(
Talk
)
>
SlidesLive Video |
Suvrit Sra 馃敆 |
Fri 9:20 a.m. - 9:30 a.m.
|
Live Q&A with Suvrit Sra (Zoom)
(
Q&A
)
>
|
Martin Takac 馃敆 |
Fri 9:45 a.m. - 10:50 a.m.
|
Poster Session 2 (gather.town) ( Poster session ) > link |
26 presentersSharan Vaswani 路 Nicolas Loizou 路 Wenjie Li 路 Preetum Nakkiran 路 Zhan Gao 路 Sina Baghal 路 Jingfeng Wu 路 Roozbeh Yousefzadeh 路 Jinyi Wang 路 Jing Wang 路 Cong Xie 路 Anastasia Borovykh 路 Stanislaw Jastrzebski 路 Soham Dan 路 Yiliang Zhang 路 Mark Tuddenham 路 Sarath Pattathil 路 Ievgen Redko 路 Jeremy Cohen 路 Yasaman Esfandiari 路 Zhanhong Jiang 路 Mostafa ElAraby 路 Chulhee Yun 路 Michael Psenka 路 Robert Gower 路 Xiaoyu Wang |
Fri 10:50 a.m. - 11:00 a.m.
|
Welcome remarks to Session 3
(
Opening remarks
)
>
|
Mark Schmidt 馃敆 |
Fri 11:00 a.m. - 11:20 a.m.
|
Invited speaker: Stochastic Geodesic Optimization, Ashia Wilson
(
Talk
)
>
|
Ashia Wilson 馃敆 |
Fri 11:20 a.m. - 11:30 a.m.
|
Live Q&A with Ashia Wilson (Zoom)
(
Q&A
)
>
|
Mark Schmidt 馃敆 |
Fri 11:30 a.m. - 11:50 a.m.
|
Invited speaker: Concentration for matrix products, and convergence of Oja鈥檚 algorithm for streaming PCA, Rachel Ward
(
Talk
)
>
|
Rachel Ward 馃敆 |
Fri 11:50 a.m. - 12:00 p.m.
|
Live Q&A with Rachel Ward (Zoom)
(
Q&A
)
>
|
Mark Schmidt 馃敆 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed talks in Session 3 (Zoom)
(
Multiple talks
)
>
|
Mark Schmidt 路 Zhan Gao 路 Wenjie Li 路 Preetum Nakkiran 路 Denny Wu 路 Chengrun Yang 馃敆 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed Video: Variance Reduction on Adaptive Stochastic Mirror Descent, Wenjie Li
(
Talk
)
>
link
SlidesLive Video |
Wenjie Li 馃敆 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed Video: Learning Rate Annealing Can Provably Help Generalization, Even for Convex Problems, Preetum Nakkiran
(
Talk
)
>
link
SlidesLive Video |
Preetum Nakkiran 馃敆 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed Video: When Does Preconditioning Help or Hurt Generalization?, Denny Wu
(
Talk
)
>
link
SlidesLive Video |
Denny Wu 馃敆 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed Video: Incremental Greedy BFGS: An Incremental Quasi-Newton Method with Explicit Superlinear Rate, Zhan Gao
(
Talk
)
>
link
SlidesLive Video |
Zhan Gao 馃敆 |
Fri 12:00 p.m. - 12:30 p.m.
|
Contributed Video: TenIPS: Inverse Propensity Sampling for Tensor Completion, Chengrun Yang
(
Talk
)
>
link
SlidesLive Video |
Chengrun Yang 馃敆 |
Fri 12:30 p.m. - 1:30 p.m.
|
Break (gather.town) link | 馃敆 |
Fri 1:30 p.m. - 1:50 p.m.
|
Invited speaker: Fast convergence of stochastic subgradient method under interpolation, Michael Friedlander
(
Talk
)
>
SlidesLive Video |
Michael Friedlander 馃敆 |
Fri 1:30 p.m. - 1:35 p.m.
|
Intro to Invited Speaker 8
(
Organizer intro
)
>
|
Mark Schmidt 馃敆 |
Fri 1:50 p.m. - 2:00 p.m.
|
Live Q&A with Michael Friedlander (Zoom)
(
Q&A
)
>
|
Mark Schmidt 馃敆 |
Fri 2:00 p.m. - 2:50 p.m.
|
Poster Session 3 (gather.town) ( Poster session ) > link |
23 presentersDenny Wu 路 Chengrun Yang 路 Tolga Ergen 路 sanae lotfi 路 Charles Guille-Escuret 路 Boris Ginsburg 路 Hanbake Lyu 路 Cong Xie 路 David Newton 路 Debraj Basu 路 Yewen Wang 路 James Lucas 路 MAOJIA LI 路 Lijun Ding 路 Jose Javier Gonzalez Ortiz 路 Reyhane Askari Hemmat 路 Zhiqi Bu 路 Neal Lawton 路 Kiran Thekumparampil 路 Jiaming Liang 路 Lindon Roberts 路 Jingyi Zhu 路 Dongruo Zhou |
Fri 2:50 p.m. - 3:00 p.m.
|
Welcome remarks to Session 4
(
Opening remarks
)
>
|
Quanquan Gu 馃敆 |
Fri 3:00 p.m. - 3:20 p.m.
|
Invited speaker: Online nonnegative matrix factorization for Markovian and other real data, Deanna Needell and Hanbaek Lyu
(
Talk
)
>
SlidesLive Video |
Hanbake Lyu 路 Deanna Needell 馃敆 |
Fri 3:20 p.m. - 3:30 p.m.
|
Live Q&A with Deanna Needell and Hanbake Lyu (Zoom)
(
Q&A
)
>
|
Quanquan Gu 馃敆 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed talks in Session 4 (Zoom)
(
Multiple talks
)
>
|
Quanquan Gu 路 sanae lotfi 路 Charles Guille-Escuret 路 Tolga Ergen 路 Dongruo Zhou 馃敆 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed Video: Stochastic Damped L-BFGS with controlled norm of the Hessian approximation, Sanae Lotfi
(
Talk
)
>
link
SlidesLive Video |
sanae lotfi 馃敆 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed Video: A Study of Condition Numbers for First-Order Optimization, Charles Guille-Escuret
(
Talk
)
>
link
SlidesLive Video |
Charles Guille-Escuret 馃敆 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed Video: Affine-Invariant Analysis of Frank-Wolfe on Strongly Convex Sets, Lewis Liu
(
Talk
)
>
link
SlidesLive Video |
馃敆 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed Video: Convex Programs for Global Optimization of Convolutional Neural Networks in Polynomial-Time, Tolga Ergen
(
Talk
)
>
link
SlidesLive Video |
Tolga Ergen 馃敆 |
Fri 3:30 p.m. - 4:00 p.m.
|
Contributed Video: On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization, Dongruo Zhou
(
Talk
)
>
link
SlidesLive Video |
Dongruo Zhou 馃敆 |
Fri 4:00 p.m. - 4:05 p.m.
|
Closing remarks
(
Closing remarks
)
>
|
Quanquan Gu 路 Courtney Paquette 路 Mark Schmidt 路 Sebastian Stich 路 Martin Takac 馃敆 |