Timezone: »

 
Poster
Fast Training Method for Stochastic Compositional Optimization Problems
Hongchang Gao · Heng Huang

Thu Dec 09 04:30 PM -- 06:00 PM (PST) @
in Poster Session 7 »

The stochastic compositional optimization problem covers a wide range of machine learning models, such as sparse additive models and model-agnostic meta-learning. Thus, it is necessary to develop efficient methods for its optimization. Existing methods for the stochastic compositional optimization problem only focus on the single machine scenario, which is far from satisfactory when data are distributed on different devices. To address this problem, we propose novel decentralized stochastic compositional gradient descent methods to efficiently train the large-scale stochastic compositional optimization problem. To the best of our knowledge, our work is the first one facilitating decentralized training for this kind of problem. Furthermore, we provide the convergence analysis for our methods, which shows that the convergence rate of our methods can achieve linear speedup with respect to the number of devices. At last, we apply our decentralized training methods to the model-agnostic meta-learning problem, and the experimental results confirm the superior performance of our methods.

Keywords: Optimization · Machine Learning · Meta Learning

Author Information

Hongchang Gao (Temple University)
Heng Huang (University of Pittsburgh)

More from the Same Authors