Timezone: »

Optimal Stochastic and Online Learning with Individual Iterates
Yunwen Lei · Peng Yang · Ke Tang · Ding-Xuan Zhou

Wed Dec 11 04:50 PM -- 04:55 PM (PST) @ West Exhibition Hall A

Stochastic composite mirror descent (SCMD) is a simple and efficient method able to capture both geometric and composite structures of optimization problems in machine learning. Existing strategies require to take either an average or a random selection of iterates to achieve optimal convergence rates, which, however, can either destroy the sparsity of solutions or slow down the practical training speed. In this paper, we propose a theoretically sound strategy to select an individual iterate of the vanilla SCMD, which is able to achieve optimal rates for both convex and strongly convex problems in a non-smooth learning setting. This strategy of outputting an individual iterate can preserve the sparsity of solutions which is crucial for a proper interpretation in sparse learning problems. We report experimental comparisons with several baseline methods to show the effectiveness of our method in achieving a fast training speed as well as in outputting sparse solutions.

Author Information

Yunwen Lei (Technical University of Kaiserslautern)
Peng Yang (Southern University of Science and Technology)
Ke Tang (Southern University of Science and Technology)
Ding-Xuan Zhou (City University of Hong Kong)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors