Timezone: »
Recently, different machine learning methods have been introduced to tackle the challenging few-shot learning scenario that is, learning from a small labeled dataset related to a specific task. Common approaches have taken the form of meta-learning: learning to learn on the new problem given the old. Following the recognition that meta-learning is implementing learning in a multi-level model, we present a Bayesian treatment for the meta-learning inner loop through the use of deep kernels. As a result we can learn a kernel that transfers to new tasks; we call this Deep Kernel Transfer (DKT). This approach has many advantages: is straightforward to implement as a single optimizer, provides uncertainty quantification, and does not require estimation of task-specific parameters. We empirically demonstrate that DKT outperforms several state-of-the-art algorithms in few-shot classification, and is the state of the art for cross-domain adaptation and regression. We conclude that complex meta-learning routines can be replaced by a simpler Bayesian model without loss of accuracy.
Author Information
Massimiliano Patacchiola (University of Edinburgh)
Massimiliano is a postdoctoral researcher at the University of Edinburgh in the Machine Learning group - School of Informatics. He is interested in efficient learning (few-shot, self-supervised, meta-learning), Bayesian methods (Gaussian processes), and (when in the right mood) robotics. Previously he has been an intern at Snapchat and a PhD student at the University of Plymouth.
Jack Turner (University of Edinburgh)
Elliot Crowley (University of Edinburgh)
Michael O'Boyle (University of Edinburgh)
Amos Storkey (University of Edinburgh)
Related Events (a corresponding poster, oral, or spotlight)
-
2020 Spotlight: Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels »
Wed Dec 9th 03:20 -- 03:30 PM Room Orals & Spotlights: Continual/Meta/Misc Learning
More from the Same Authors
-
2020 Poster: Self-Supervised Relational Reasoning for Representation Learning »
Massimiliano Patacchiola · Amos Storkey -
2020 Spotlight: Self-Supervised Relational Reasoning for Representation Learning »
Massimiliano Patacchiola · Amos Storkey -
2019 Poster: Zero-shot Knowledge Transfer via Adversarial Belief Matching »
Paul Micaelli · Amos Storkey -
2019 Spotlight: Zero-shot Knowledge Transfer via Adversarial Belief Matching »
Paul Micaelli · Amos Storkey -
2019 Poster: Learning to Learn By Self-Critique »
Antreas Antoniou · Amos Storkey -
2018 Poster: Moonshine: Distilling with Cheap Convolutions »
Elliot Crowley · Gavin Gray · Amos Storkey -
2015 Poster: Covariance-Controlled Adaptive Langevin Thermostat for Large-Scale Bayesian Sampling »
Xiaocheng Shang · Zhanxing Zhu · Benedict Leimkuhler · Amos Storkey -
2014 Workshop: NIPS Workshop on Transactional Machine Learning and E-Commerce »
David Parkes · David H Wolpert · Jennifer Wortman Vaughan · Jacob D Abernethy · Amos Storkey · Mark Reid · Ping Jin · Nihar Bhadresh Shah · Mehryar Mohri · Luis E Ortiz · Robin Hanson · Aaron Roth · Satyen Kale · Sebastien Lahaie -
2012 Poster: Continuous Relaxations for Discrete Hamiltonian Monte Carlo »
Zoubin Ghahramani · Yichuan Zhang · Charles Sutton · Amos Storkey -
2012 Spotlight: Continuous Relaxations for Discrete Hamiltonian Monte Carlo »
Zoubin Ghahramani · Yichuan Zhang · Charles Sutton · Amos Storkey -
2012 Poster: The Coloured Noise Expansion and Parameter Estimation of Diffusion Processes »
Simon Lyons · Amos Storkey · Simo Sarkka -
2011 Poster: Neuronal Adaptation for Sampling-Based Probabilistic Inference in Perceptual Bistability »
David Reichert · Peggy Series · Amos Storkey -
2011 Spotlight: Neuronal Adaptation for Sampling-Based Probabilistic Inference in Perceptual Bistability »
David Reichert · Peggy Series · Amos Storkey -
2010 Poster: Hallucinations in Charles Bonnet Syndrome Induced by Homeostasis: a Deep Boltzmann Machine Model »
David Reichert · Peggy Series · Amos Storkey -
2010 Poster: Sparse Instrumental Variables (SPIV) for Genome-Wide Studies »
Felix V Agakov · Paul McKeigue · Jon Krohn · Amos Storkey -
2007 Poster: Continuous Time Particle Filtering for fMRI »
Lawrence Murray · Amos Storkey -
2007 Poster: Modelling motion primitives and their timing in biologically executed movements »
Ben H Williams · Marc Toussaint · Amos Storkey -
2006 Poster: Learning Structural Equation Models for fMRI »
Amos Storkey · Enrico Simonotto · Heather Whalley · Stephen Lawrie · Lawrence Murray · David McGonigle -
2006 Poster: Mixture Regression for Covariate Shift »
Amos Storkey · Masashi Sugiyama