Timezone: »

 
Poster
Online Adaptive Methods, Universality and Acceleration
Kfir Y. Levy · Alp Yurtsever · Volkan Cevher

Thu Dec 06 02:00 PM -- 04:00 PM (PST) @ Room 517 AB #133
in Thu Poster Session B »

We present a novel method for convex unconstrained optimization that, without any modifications ensures: (1) accelerated convergence rate for smooth objectives, (2) standard convergence rate in the general (non-smooth) setting, and (3) standard convergence rate in the stochastic optimization setting.
To the best of our knowledge, this is the first method that simultaneously applies to all of the above settings.
At the heart of our method is an adaptive learning rate rule that employs importance weights, in the spirit of adaptive online learning algorithms [duchi2011adaptive,levy2017online], combined with an update that linearly couples two sequences, in the spirit of [AllenOrecchia2017]. An empirical examination of our method demonstrates its applicability to the above mentioned scenarios and corroborates our theoretical findings.

Keywords: Online Learning
[ Paper] 

Author Information

Kfir Y. Levy (ETH)
Alp Yurtsever (EPFL)
Volkan Cevher (EPFL)

More from the Same Authors