Timezone: »
Few-shot learning is a central problem in meta-learning, where learners must quickly adapt to new tasks given limited training data. Recently, feature pre-training has become a ubiquitous component in state-of-the-art meta-learning methods and is shown to provide significant performance improvement. However, there is limited theoretical understanding of the connection between pre-training and meta-learning. Further, pre-training requires global labels shared across tasks, which may be unavailable in practice. In this paper, we show why exploiting pre-training is theoretically advantageous for meta-learning, and in particular the critical role of global labels. This motivates us to propose Meta Label Learning (MeLa), a novel meta-learning framework that automatically infers global labels to obtains robust few-shot models. Empirically, we demonstrate that MeLa is competitive with existing methods and provide extensive ablation experiments to highlight its key properties.
Author Information
Ruohan Wang (Institute for Infocomm Research)
Massimiliano Pontil (IIT & UCL)
Carlo Ciliberto (University College London)
More from the Same Authors
-
2021 : Linear Convergence of Batch Greenkhorn for Regularized Multimarginal Optimal Transport »
Vladimir Kostic · Saverio Salzo · Massimiliano Pontil -
2022 Poster: Conditional Meta-Learning of Linear Representations »
Giulia Denevi · Massimiliano Pontil · Carlo Ciliberto -
2022 Spotlight: Conditional Meta-Learning of Linear Representations »
Giulia Denevi · Massimiliano Pontil · Carlo Ciliberto -
2022 Spotlight: Lightning Talks 3B-1 »
Tianying Ji · Tongda Xu · Giulia Denevi · Aibek Alanov · Martin Wistuba · Wei Zhang · Yuesong Shen · Massimiliano Pontil · Vadim Titov · Yan Wang · Yu Luo · Daniel Cremers · Yanjun Han · Arlind Kadra · Dailan He · Josif Grabocka · Zhengyuan Zhou · Fuchun Sun · Carlo Ciliberto · Dmitry Vetrov · Mingxuan Jing · Chenjian Gao · Aaron Flores · Tsachy Weissman · Han Gao · Fengxiang He · Kunzan Liu · Wenbing Huang · Hongwei Qin -
2022 Spotlight: A gradient estimator via L1-randomization for online zero-order optimization with two point feedback »
Arya Akhavan · Evgenii Chzhen · Massimiliano Pontil · Alexandre Tsybakov -
2022 Poster: A gradient estimator via L1-randomization for online zero-order optimization with two point feedback »
Arya Akhavan · Evgenii Chzhen · Massimiliano Pontil · Alexandre Tsybakov -
2022 Poster: Learning Dynamical Systems via Koopman Operator Regression in Reproducing Kernel Hilbert Spaces »
Vladimir Kostic · Pietro Novelli · Andreas Maurer · Carlo Ciliberto · Lorenzo Rosasco · Massimiliano Pontil -
2022 Poster: Group Meritocratic Fairness in Linear Contextual Bandits »
Riccardo Grazzi · Arya Akhavan · John IF Falk · Leonardo Cella · Massimiliano Pontil -
2021 : Carlo Ciliberto Q&A »
Carlo Ciliberto -
2021 : Carlo Ciliberto »
Carlo Ciliberto -
2021 Poster: Concentration inequalities under sub-Gaussian and sub-exponential conditions »
Andreas Maurer · Massimiliano Pontil -
2021 Poster: A Gang of Adversarial Bandits »
Mark Herbster · Stephen Pasteris · Fabio Vitale · Massimiliano Pontil -
2021 Poster: PSD Representations for Effective Probability Models »
Alessandro Rudi · Carlo Ciliberto -
2021 Poster: Distributed Zero-Order Optimization under Adversarial Noise »
Arya Akhavan · Massimiliano Pontil · Alexandre Tsybakov -
2020 Poster: The Advantage of Conditional Meta-Learning for Biased Regularization and Fine Tuning »
Giulia Denevi · Massimiliano Pontil · Carlo Ciliberto -
2020 Poster: Structured Prediction for Conditional Meta-Learning »
Ruohan Wang · Yiannis Demiris · Carlo Ciliberto -
2020 Poster: Exploiting MMD and Sinkhorn Divergences for Fair and Transferable Representation Learning »
Luca Oneto · Michele Donini · Giulia Luise · Carlo Ciliberto · Andreas Maurer · Massimiliano Pontil -
2020 Poster: Fair regression with Wasserstein barycenters »
Evgenii Chzhen · Christophe Denis · Mohamed Hebiri · Luca Oneto · Massimiliano Pontil -
2020 Poster: Fair regression via plug-in estimator and recalibration with statistical guarantees »
Evgenii Chzhen · Christophe Denis · Mohamed Hebiri · Luca Oneto · Massimiliano Pontil -
2020 Oral: Fair regression via plug-in estimator and recalibration with statistical guarantees »
Evgenii Chzhen · Christophe Denis · Mohamed Hebiri · Luca Oneto · Massimiliano Pontil -
2019 : Poster Session »
Rishav Chourasia · Yichong Xu · Corinna Cortes · Chien-Yi Chang · Yoshihiro Nagano · So Yeon Min · Benedikt Boecking · Phi Vu Tran · Kamyar Ghasemipour · Qianggang Ding · Shouvik Mani · Vikram Voleti · Rasool Fakoor · Miao Xu · Kenneth Marino · Lisa Lee · Volker Tresp · Jean-Francois Kagy · Marvin Zhang · Barnabas Poczos · Dinesh Khandelwal · Adrien Bardes · Evan Shelhamer · Jiacheng Zhu · Ziming Li · Xiaoyan Li · Dmitrii Krasheninnikov · Ruohan Wang · Mayoore Jaiswal · Emad Barsoum · Suvansh Sanjeev · Theeraphol Wattanavekin · Qizhe Xie · Sifan Wu · Yuki Yoshida · David Kanaa · Sina Khoshfetrat Pakazad · Mehdi Maasoumy -
2019 Poster: Leveraging Labeled and Unlabeled Data for Consistent Fair Binary Classification »
Evgenii Chzhen · Christophe Denis · Mohamed Hebiri · Luca Oneto · Massimiliano Pontil -
2019 Poster: Online-Within-Online Meta-Learning »
Giulia Denevi · Dimitris Stamos · Carlo Ciliberto · Massimiliano Pontil -
2019 Poster: Localized Structured Prediction »
Carlo Ciliberto · Francis Bach · Alessandro Rudi -
2019 Poster: Sinkhorn Barycenters with Free Support via Frank-Wolfe Algorithm »
Giulia Luise · Saverio Salzo · Massimiliano Pontil · Carlo Ciliberto -
2019 Spotlight: Sinkhorn Barycenters with Free Support via Frank-Wolfe Algorithm »
Giulia Luise · Saverio Salzo · Massimiliano Pontil · Carlo Ciliberto -
2018 Poster: Learning To Learn Around A Common Mean »
Giulia Denevi · Carlo Ciliberto · Dimitris Stamos · Massimiliano Pontil -
2018 Poster: Differential Properties of Sinkhorn Approximation for Learning with Wasserstein Distance »
Giulia Luise · Alessandro Rudi · Massimiliano Pontil · Carlo Ciliberto -
2018 Poster: Empirical Risk Minimization Under Fairness Constraints »
Michele Donini · Luca Oneto · Shai Ben-David · John Shawe-Taylor · Massimiliano Pontil -
2018 Poster: Manifold Structured Prediction »
Alessandro Rudi · Carlo Ciliberto · Gian Maria Marconi · Lorenzo Rosasco -
2017 Poster: Consistent Multitask Learning with Nonlinear Output Relations »
Carlo Ciliberto · Alessandro Rudi · Lorenzo Rosasco · Massimiliano Pontil -
2016 Poster: A Consistent Regularization Approach for Structured Prediction »
Carlo Ciliberto · Lorenzo Rosasco · Alessandro Rudi -
2013 Workshop: Output Representation Learning »
Yuhong Guo · Dale Schuurmans · Richard Zemel · Samy Bengio · Yoshua Bengio · Li Deng · Dan Roth · Kilian Q Weinberger · Jason Weston · Kihyuk Sohn · Florent Perronnin · Gabriel Synnaeve · Pablo R Strasser · julien audiffren · Carlo Ciliberto · Dan Goldwasser