Timezone: »
The ever-increasing size of modern data sets combined with the difficulty of obtaining label information has made semi-supervised learning one of the problems of significant practical importance in modern data analysis. We revisit the approach to semi-supervised learning with generative models and develop new models that allow for effective generalisation from small labelled data sets to large unlabelled ones. Generative approaches have thus far been either inflexible, inefficient or non-scalable. We show that deep generative models and approximate Bayesian inference exploiting recent advances in variational methods can be used to provide significant improvements, making generative approaches highly competitive for semi-supervised learning.
Author Information
Diederik Kingma (Google)
Shakir Mohamed (DeepMind)
Shakir Mohamed is a senior staff scientist at DeepMind in London. Shakir's main interests lie at the intersection of approximate Bayesian inference, deep learning and reinforcement learning, and the role that machine learning systems at this intersection have in the development of more intelligent and general-purpose learning systems. Before moving to London, Shakir held a Junior Research Fellowship from the Canadian Institute for Advanced Research (CIFAR), based in Vancouver at the University of British Columbia with Nando de Freitas. Shakir completed his PhD with Zoubin Ghahramani at the University of Cambridge, where he was a Commonwealth Scholar to the United Kingdom. Shakir is from South Africa and completed his previous degrees in Electrical and Information Engineering at the University of the Witwatersrand, Johannesburg.
Danilo Jimenez Rezende (Google DeepMind)
Max Welling (University of Amsterdam / Qualcomm AI Research)
Related Events (a corresponding poster, oral, or spotlight)
-
2014 Spotlight: Semi-supervised Learning with Deep Generative Models »
Tue Dec 9th 10:40 -- 11:00 PM Room Level 2, room 210
More from the Same Authors
-
2020 Poster: Natural Graph Networks »
Pim de Haan · Taco Cohen · Max Welling -
2020 Poster: SE(3)-Transformers: 3D Roto-Translation Equivariant Attention Networks »
Fabian Fuchs · Daniel E Worrall · Volker Fischer · Max Welling -
2020 Poster: SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows »
Didrik Nielsen · Priyank Jaini · Emiel Hoogeboom · Ole Winther · Max Welling -
2020 Oral: SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows »
Didrik Nielsen · Priyank Jaini · Emiel Hoogeboom · Ole Winther · Max Welling -
2020 Poster: ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA »
Ilyes Khemakhem · Ricardo Monti · Diederik Kingma · Aapo Hyvarinen -
2020 Poster: The Convolution Exponential and Generalized Sylvester Flows »
Emiel Hoogeboom · Victor Garcia Satorras · Jakub Tomczak · Max Welling -
2020 Poster: Bayesian Bits: Unifying Quantization and Pruning »
Mart van Baalen · Christos Louizos · Markus Nagel · Rana Ali Amjad · Ying Wang · Tijmen Blankevoort · Max Welling -
2020 Poster: Experimental design for MRI by greedy policy search »
Tim Bakker · Herke van Hoof · Max Welling -
2020 Spotlight: Experimental design for MRI by greedy policy search »
Tim Bakker · Herke van Hoof · Max Welling -
2020 Spotlight: ICE-BeeM: Identifiable Conditional Energy-Based Deep Models Based on Nonlinear ICA »
Ilyes Khemakhem · Ricardo Monti · Diederik Kingma · Aapo Hyvarinen -
2020 Poster: MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning »
Elise van der Pol · Daniel E Worrall · Herke van Hoof · Frans Oliehoek · Max Welling -
2020 AffinityWorkshop: Muslims in ML »
Marzyeh Ghassemi · Mohammad Norouzi · Shakir Mohamed · Aya Salama · Tasmie Sarker -
2019 Workshop: Bayesian Deep Learning »
Yarin Gal · José Miguel Hernández-Lobato · Christos Louizos · Eric Nalisnick · Zoubin Ghahramani · Kevin Murphy · Max Welling -
2019 Poster: Invert to Learn to Invert »
Patrick Putzky · Max Welling -
2019 Poster: Training Language GANs from Scratch »
Cyprien de Masson d'Autume · Shakir Mohamed · Mihaela Rosca · Jack Rae -
2019 Poster: Deep Scale-spaces: Equivariance Over Scale »
Daniel Worrall · Max Welling -
2019 Poster: Towards Interpretable Reinforcement Learning Using Attention Augmented Agents »
Alexander Mott · Daniel Zoran · Mike Chrzanowski · Daan Wierstra · Danilo Jimenez Rezende -
2019 Poster: Shaping Belief States with Generative Environment Models for RL »
Karol Gregor · Danilo Jimenez Rezende · Frederic Besse · Yan Wu · Hamza Merzic · Aaron van den Oord -
2019 Poster: Integer Discrete Flows and Lossless Compression »
Emiel Hoogeboom · Jorn Peters · Rianne van den Berg · Max Welling -
2019 Poster: The Functional Neural Process »
Christos Louizos · Xiahan Shi · Klamer Schutte · Max Welling -
2019 Poster: Combining Generative and Discriminative Models for Hybrid Inference »
Victor Garcia Satorras · Zeynep Akata · Max Welling -
2019 Spotlight: Combining Generative and Discriminative Models for Hybrid Inference »
Victor Garcia Satorras · Max Welling · Zeynep Akata -
2019 Poster: Combinatorial Bayesian Optimization using the Graph Cartesian Product »
Changyong Oh · Jakub Tomczak · Efstratios Gavves · Max Welling -
2018 Workshop: Bayesian Deep Learning »
Yarin Gal · José Miguel Hernández-Lobato · Christos Louizos · Andrew Wilson · Zoubin Ghahramani · Kevin Murphy · Max Welling -
2018 Workshop: NIPS 2018 workshop on Compact Deep Neural Networks with industrial applications »
Lixin Fan · Zhouchen Lin · Max Welling · Yurong Chen · Werner Bailer -
2018 Poster: Implicit Reparameterization Gradients »
Mikhail Figurnov · Shakir Mohamed · Andriy Mnih -
2018 Spotlight: Implicit Reparameterization Gradients »
Mikhail Figurnov · Shakir Mohamed · Andriy Mnih -
2018 Poster: A Probabilistic U-Net for Segmentation of Ambiguous Images »
Simon Kohl · Bernardino Romera-Paredes · Clemens Meyer · Jeffrey De Fauw · Joseph R. Ledsam · Klaus Maier-Hein · S. M. Ali Eslami · Danilo Jimenez Rezende · Olaf Ronneberger -
2018 Spotlight: A Probabilistic U-Net for Segmentation of Ambiguous Images »
Simon Kohl · Bernardino Romera-Paredes · Clemens Meyer · Jeffrey De Fauw · Joseph R. Ledsam · Klaus Maier-Hein · S. M. Ali Eslami · Danilo Jimenez Rezende · Olaf Ronneberger -
2018 Poster: Graphical Generative Adversarial Networks »
Chongxuan LI · Max Welling · Jun Zhu · Bo Zhang -
2018 Poster: Glow: Generative Flow with Invertible 1x1 Convolutions »
Diederik Kingma · Prafulla Dhariwal -
2018 Poster: 3D Steerable CNNs: Learning Rotationally Equivariant Features in Volumetric Data »
Maurice Weiler · Wouter Boomsma · Mario Geiger · Max Welling · Taco Cohen -
2017 Workshop: Bayesian Deep Learning »
Yarin Gal · José Miguel Hernández-Lobato · Christos Louizos · Andrew Wilson · Andrew Wilson · Diederik Kingma · Zoubin Ghahramani · Kevin Murphy · Max Welling -
2017 Workshop: Advances in Approximate Bayesian Inference »
Francisco Ruiz · Stephan Mandt · Cheng Zhang · James McInerney · James McInerney · Dustin Tran · Dustin Tran · David Blei · Max Welling · Tamara Broderick · Michalis Titsias -
2017 Poster: Imagination-Augmented Agents for Deep Reinforcement Learning »
Sébastien Racanière · Theophane Weber · David Reichert · Lars Buesing · Arthur Guez · Danilo Jimenez Rezende · Adrià Puigdomènech Badia · Oriol Vinyals · Nicolas Heess · Yujia Li · Razvan Pascanu · Peter Battaglia · Demis Hassabis · David Silver · Daan Wierstra -
2017 Poster: Causal Effect Inference with Deep Latent-Variable Models »
Christos Louizos · Uri Shalit · Joris M Mooij · David Sontag · Richard Zemel · Max Welling -
2017 Oral: Imagination-Augmented Agents for Deep Reinforcement Learning »
Sébastien Racanière · Theophane Weber · David Reichert · Lars Buesing · Arthur Guez · Danilo Jimenez Rezende · Adrià Puigdomènech Badia · Oriol Vinyals · Nicolas Heess · Yujia Li · Razvan Pascanu · Peter Battaglia · Demis Hassabis · David Silver · Daan Wierstra -
2017 Poster: Variational Memory Addressing in Generative Models »
Jörg Bornschein · Andriy Mnih · Daniel Zoran · Danilo Jimenez Rezende -
2017 Poster: Bayesian Compression for Deep Learning »
Christos Louizos · Karen Ullrich · Max Welling -
2016 Workshop: Bayesian Deep Learning »
Yarin Gal · Christos Louizos · Zoubin Ghahramani · Kevin Murphy · Max Welling -
2016 Poster: Unsupervised Learning of 3D Structure from Images »
Danilo Jimenez Rezende · S. M. Ali Eslami · Shakir Mohamed · Peter Battaglia · Max Jaderberg · Nicolas Heess -
2016 Poster: Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks »
Tim Salimans · Diederik Kingma -
2016 Oral: Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks »
Tim Salimans · Diederik Kingma -
2016 Poster: Towards Conceptual Compression »
Karol Gregor · Frederic Besse · Danilo Jimenez Rezende · Ivo Danihelka · Daan Wierstra -
2016 Poster: Interaction Networks for Learning about Objects, Relations and Physics »
Peter Battaglia · Razvan Pascanu · Matthew Lai · Danilo Jimenez Rezende · koray kavukcuoglu -
2016 Poster: Improving Variational Autoencoders with Inverse Autoregressive Flow »
Diederik Kingma · Tim Salimans · Rafal Jozefowicz · Peter Chen · Xi Chen · Ilya Sutskever · Max Welling -
2016 Tutorial: Variational Inference: Foundations and Modern Methods »
David Blei · Shakir Mohamed · Rajesh Ranganath -
2015 Workshop: Scalable Monte Carlo Methods for Bayesian Analysis of Big Data »
Babak Shahbaba · Yee Whye Teh · Max Welling · Arnaud Doucet · Christophe Andrieu · Sebastian J. Vollmer · Pierre Jacob -
2015 Workshop: Advances in Approximate Bayesian Inference »
Dustin Tran · Tamara Broderick · Stephan Mandt · James McInerney · Shakir Mohamed · Alp Kucukelbir · Matthew D. Hoffman · Neil Lawrence · David Blei -
2015 Symposium: Deep Learning Symposium »
Yoshua Bengio · Marc'Aurelio Ranzato · Honglak Lee · Max Welling · Andrew Y Ng -
2015 Poster: Bayesian dark knowledge »
Anoop Korattikara Balan · Vivek Rathod · Kevin Murphy · Max Welling -
2015 Poster: Optimization Monte Carlo: Efficient and Embarrassingly Parallel Likelihood-Free Inference »
Ted Meeds · Max Welling -
2015 Poster: Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning »
Shakir Mohamed · Danilo Jimenez Rezende -
2015 Poster: Variational Dropout and the Local Reparameterization Trick »
Diederik Kingma · Tim Salimans · Max Welling -
2014 Workshop: Advances in Variational Inference »
David Blei · Shakir Mohamed · Michael Jordan · Charles Blundell · Tamara Broderick · Matthew D. Hoffman -
2014 Workshop: ABC in Montreal »
Max Welling · Neil D Lawrence · Richard D Wilkinson · Ted Meeds · Christian X Robert -
2014 Demonstration: Machine Learning in the Browser »
Ted Meeds · Remco Hendriks · Said Al Faraby · Magiel Bruntink · Max Welling -
2013 Workshop: Probabilistic Models for Big Data »
Neil D Lawrence · Joaquin Quiñonero Candela · Tianshi Gao · James Hensman · Zoubin Ghahramani · Max Welling · David Blei · Ralf Herbrich -
2012 Workshop: Bayesian Optimization and Decision Making »
Javad Azimi · Roman Garnett · Frank R Hutter · Shakir Mohamed -
2012 Poster: Expectation Propagation in Gaussian Process Dynamical Systems »
Marc Deisenroth · Shakir Mohamed -
2012 Poster: Fast Bayesian Inference for Non-Conjugate Gaussian Process Regression »
Mohammad Emtiyaz Khan · Shakir Mohamed · Kevin P Murphy -
2012 Poster: The Time-Marginalized Coalescent Prior for Hierarchical Clustering »
Levi Boyles · Max Welling -
2011 Poster: Statistical Tests for Optimization Efficiency »
Levi Boyles · Anoop Korattikara · Deva Ramanan · Max Welling -
2010 Poster: On Herding and the Perceptron Cycling Theorem »
Andrew E Gelfand · Yutian Chen · Laurens van der Maaten · Max Welling -
2010 Poster: Regularized estimation of image statistics by Score Matching »
Diederik Kingma · Yann LeCun -
2009 Poster: Large Scale Nonparametric Bayesian Inference: Data Parallelisation in the Indian Buffet Process »
Shakir Mohamed · David A Knowles · Zoubin Ghahramani · Finale P Doshi-Velez -
2008 Session: Oral session 10: Nonparametric Processes, Scene Processing and Image Statistics »
Max Welling -
2008 Poster: Bayesian Exponential Family PCA »
Shakir Mohamed · Katherine Heller · Zoubin Ghahramani -
2008 Poster: Asynchronous Distributed Learning of Topic Models »
Arthur Asuncion · Padhraic Smyth · Max Welling -
2008 Spotlight: Bayesian Exponential Family PCA »
Shakir Mohamed · Katherine Heller · Zoubin Ghahramani -
2007 Spotlight: Collapsed Variational Inference for HDP »
Yee Whye Teh · Kenichi Kurihara · Max Welling -
2007 Spotlight: Distributed Inference for Latent Dirichlet Allocation »
David Newman · Arthur Asuncion · Padhraic Smyth · Max Welling -
2007 Poster: Infinite State Bayes-Nets for Structured Domains »
Max Welling · Ian Porteous · Evgeniy Bart -
2007 Poster: Collapsed Variational Inference for HDP »
Yee Whye Teh · Kenichi Kurihara · Max Welling -
2007 Poster: Distributed Inference for Latent Dirichlet Allocation »
David Newman · Arthur Asuncion · Padhraic Smyth · Max Welling -
2007 Spotlight: Infinite State Bayes-Nets for Structured Domains »
Max Welling · Ian Porteous · Evgeniy Bart -
2006 Poster: Structure Learning in Markov Random Fields »
Sridevi Parise · Max Welling -
2006 Poster: Accelerated Variational Dirichlet Process Mixtures »
Kenichi Kurihara · Max Welling · Nikos Vlassis -
2006 Spotlight: Accelerated Variational Dirichlet Process Mixtures »
Kenichi Kurihara · Max Welling · Nikos Vlassis -
2006 Poster: A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation »
Yee Whye Teh · David Newman · Max Welling