Poster

GAGA: Deciphering Age-path of Generalized Self-paced Regularizer

Xingyu Qu · Diyang Li · Xiaohan Zhao · Bin Gu

Keywords: [ self-paced learning ] [ partial optimum ] [ biconvex optimization ] [ solution path ]

[ Abstract ]
[ Poster [ OpenReview
 
Spotlight presentation: Lightning Talks 4A-3
Wed 7 Dec 6 p.m. PST — 6:15 p.m. PST

Abstract:

Nowadays self-paced learning (SPL) is an important machine learning paradigm that mimics the cognitive process of humans and animals. The SPL regime involves a self-paced regularizer and a gradually increasing age parameter, which plays a key role in SPL but where to optimally terminate this process is still non-trivial to determine. A natural idea is to compute the solution path w.r.t. age parameter (i.e., age-path). However, current age-path algorithms are either limited to the simplest regularizer, or lack solid theoretical understanding as well as computational efficiency. To address this challenge, we propose a novel Generalized Age-path Algorithm (GAGA) for SPL with various self-paced regularizers based on ordinary differential equations (ODEs) and sets control, which can learn the entire solution spectrum w.r.t. a range of age parameters. To the best of our knowledge, GAGA is the first exact path-following algorithm tackling the age-path for general self-paced regularizer. Finally the algorithmic steps of classic SVM and Lasso are described in detail. We demonstrate the performance of GAGA on real-world datasets, and find considerable speedup between our algorithm and competing baselines.

Chat is not available.