Timezone: »

 
Poster
Learning To Learn Around A Common Mean
Giulia Denevi · Carlo Ciliberto · Dimitris Stamos · Massimiliano Pontil

Thu Dec 06 02:00 PM -- 04:00 PM (PST) @ Room 517 AB #125
in Thu Poster Session B »

The problem of learning-to-learn (LTL) or meta-learning is gaining increasing attention due to recent empirical evidence of its effectiveness in applications. The goal addressed in LTL is to select an algorithm that works well on tasks sampled from a meta-distribution. In this work, we consider the family of algorithms given by a variant of Ridge Regression, in which the regularizer is the square distance to an unknown mean vector. We show that, in this setting, the LTL problem can be reformulated as a Least Squares (LS) problem and we exploit a novel meta- algorithm to efficiently solve it. At each iteration the meta-algorithm processes only one dataset. Specifically, it firstly estimates the stochastic LS objective function, by splitting this dataset into two subsets used to train and test the inner algorithm, respectively. Secondly, it performs a stochastic gradient step with the estimated value. Under specific assumptions, we present a bound for the generalization error of our meta-algorithm, which suggests the right splitting parameter to choose. When the hyper-parameters of the problem are fixed, this bound is consistent as the number of tasks grows, even if the sample size is kept constant. Preliminary experiments confirm our theoretical findings, highlighting the advantage of our approach, with respect to independent task learning.

Keywords: Online Learning · Multitask and Transfer Learning · Regression · Meta-Learning
[ Paper]  [ Poster] 

Author Information

Giulia Denevi (IIT/UNIGE)
Carlo Ciliberto (Imperial College London)
Dimitris Stamos (University College London)
Massimiliano Pontil (IIT & UCL)

More from the Same Authors