Timezone: »
Principal Components Analysis (PCA) has become established as one of the key tools for dimensionality reduction when dealing with real valued data. Approaches such as exponential family PCA and non-negative matrix factorisation have successfully extended PCA to non-Gaussian data types, but these techniques fail to take advantage of Bayesian inference and can suffer from problems of overfitting and poor generalisation. This paper presents a fully probabilistic approach to PCA, which is generalised to the exponential family, based on Hybrid Monte Carlo sampling. We describe the model which is based on a factorisation of the observed data matrix, and show performance of the model on both synthetic and real data.
Author Information
Shakir Mohamed (DeepMind)

Shakir Mohamed is a senior staff scientist at DeepMind in London. Shakir's main interests lie at the intersection of approximate Bayesian inference, deep learning and reinforcement learning, and the role that machine learning systems at this intersection have in the development of more intelligent and general-purpose learning systems. Before moving to London, Shakir held a Junior Research Fellowship from the Canadian Institute for Advanced Research (CIFAR), based in Vancouver at the University of British Columbia with Nando de Freitas. Shakir completed his PhD with Zoubin Ghahramani at the University of Cambridge, where he was a Commonwealth Scholar to the United Kingdom. Shakir is from South Africa and completed his previous degrees in Electrical and Information Engineering at the University of the Witwatersrand, Johannesburg.
Katherine Heller (Google)
Zoubin Ghahramani (Uber and University of Cambridge)
Zoubin Ghahramani is Professor of Information Engineering at the University of Cambridge, where he leads the Machine Learning Group. He studied computer science and cognitive science at the University of Pennsylvania, obtained his PhD from MIT in 1995, and was a postdoctoral fellow at the University of Toronto. His academic career includes concurrent appointments as one of the founding members of the Gatsby Computational Neuroscience Unit in London, and as a faculty member of CMU's Machine Learning Department for over 10 years. His current research interests include statistical machine learning, Bayesian nonparametrics, scalable inference, probabilistic programming, and building an automatic statistician. He has held a number of leadership roles as programme and general chair of the leading international conferences in machine learning including: AISTATS (2005), ICML (2007, 2011), and NIPS (2013, 2014). In 2015 he was elected a Fellow of the Royal Society.
Related Events (a corresponding poster, oral, or spotlight)
-
2008 Spotlight: Bayesian Exponential Family PCA »
Thu. Dec 11th 01:27 -- 01:28 AM Room
More from the Same Authors
-
2021 : Maintaining fairness across distribution shifts: do we have viable solutions for real-world applications? »
Jessica Schrouff · Natalie Harris · Sanmi Koyejo · Ibrahim Alabdulmohsin · Eva Schnider · Diana Mincu · Christina Chen · Awa Dieng · Yuan Liu · Vivek Natarajan · Katherine Heller · Alexander D'Amour -
2022 : Participatory Systems for Personalized Prediction »
Hailey James · Chirag Nagpal · Katherine Heller · Berk Ustun -
2022 : Participatory Systems for Personalized Prediction »
Hailey James · Berk Ustun · Chirag Nagpal · Katherine Heller -
2023 Poster: Participatory Personalization in Classification »
Hailey James · Chirag Nagpal · Katherine Heller · Berk Ustun -
2022 : Advancing the participatory approach to AI in Mental Health »
Wilson Lee · Munmun De Choudhury · Morgan Scheuerman · Julia Hamer-Hunt · Dan Joyce · Nenad Tomasev · Kevin McKee · Shakir Mohamed · Danielle Belgrave · Christopher Burr -
2022 : Participatory Systems for Personalized Prediction »
Hailey James · Chirag Nagpal · Katherine Heller · Berk Ustun -
2022 : Participatory Systems for Personalized Prediction »
Hailey James · Chirag Nagpal · Katherine Heller · Berk Ustun -
2022 Poster: Diagnosing failures of fairness transfer across distribution shift in real-world medical settings »
Jessica Schrouff · Natalie Harris · Sanmi Koyejo · Ibrahim Alabdulmohsin · Eva Schnider · Krista Opsahl-Ong · Alexander Brown · Subhrajit Roy · Diana Mincu · Christina Chen · Awa Dieng · Yuan Liu · Vivek Natarajan · Alan Karthikesalingam · Katherine Heller · Silvia Chiappa · Alexander D'Amour -
2021 Workshop: Bayesian Deep Learning »
Yarin Gal · Yingzhen Li · Sebastian Farquhar · Christos Louizos · Eric Nalisnick · Andrew Gordon Wilson · Zoubin Ghahramani · Kevin Murphy · Max Welling -
2021 Poster: Deep Neural Networks as Point Estimates for Deep Gaussian Processes »
Vincent Dutordoir · James Hensman · Mark van der Wilk · Carl Henrik Ek · Zoubin Ghahramani · Nicolas Durrande -
2020 : Panel Discussions »
Grace Lindsay · George Konidaris · Shakir Mohamed · Kimberly Stachenfeld · Peter Dayan · Yael Niv · Doina Precup · Catherine Hartley · Ishita Dasgupta -
2020 Workshop: Machine Learning for Mobile Health »
Joseph Futoma · Walter Dempsey · Katherine Heller · Yian Ma · Nicholas Foti · Marianne Njifon · Kelly Zhang · Jieru Shi -
2020 : Invited talk 1 QnA: Shakir Mohamed »
Shakir Mohamed · Feryal Behbahani · Raymond Chua -
2020 : Invited Talk #1 Shakir Mohamed : Pain and Machine Learning »
Shakir Mohamed -
2020 : Q&A with Shakir »
Shakir Mohamed -
2020 : Invited: Shakir Mohamed »
Shakir Mohamed -
2020 Symposium: COVID-19 Symposium Day 2 »
Andrew Beam · Tristan Naumann · Katherine Heller · Elaine Nsoesie -
2020 Symposium: COVID-19 Symposium Day 1 »
Andrew Beam · Tristan Naumann · Katherine Heller · Elaine Nsoesie -
2020 : Policy Panel »
Roya Pakzad · Dia Kayyali · Marzyeh Ghassemi · Shakir Mohamed · Mohammad Norouzi · Ted Pedersen · Anver Emon · Abubakar Abid · Darren Byler · Samhaa R. El-Beltagy · Nayel Shafei · Mona Diab -
2020 Affinity Workshop: Muslims in ML »
Marzyeh Ghassemi · Mohammad Norouzi · Shakir Mohamed · Aya Salama · Tasmie Sarker -
2019 Workshop: Bayesian Deep Learning »
Yarin Gal · José Miguel Hernández-Lobato · Christos Louizos · Eric Nalisnick · Zoubin Ghahramani · Kevin Murphy · Max Welling -
2019 Poster: Training Language GANs from Scratch »
Cyprien de Masson d'Autume · Shakir Mohamed · Mihaela Rosca · Jack Rae -
2019 Poster: Reconciling meta-learning and continual learning with online mixtures of tasks »
Ghassen Jerfel · Erin Grant · Tom Griffiths · Katherine Heller -
2019 Spotlight: Reconciling meta-learning and continual learning with online mixtures of tasks »
Ghassen Jerfel · Erin Grant · Tom Griffiths · Katherine Heller -
2019 Poster: Bayesian Learning of Sum-Product Networks »
Martin Trapp · Robert Peharz · Hong Ge · Franz Pernkopf · Zoubin Ghahramani -
2018 : Katherine Heller »
Katherine Heller -
2018 Workshop: Bayesian Deep Learning »
Yarin Gal · José Miguel Hernández-Lobato · Christos Louizos · Andrew Wilson · Zoubin Ghahramani · Kevin Murphy · Max Welling -
2018 Poster: Implicit Reparameterization Gradients »
Mikhail Figurnov · Shakir Mohamed · Andriy Mnih -
2018 Spotlight: Implicit Reparameterization Gradients »
Mikhail Figurnov · Shakir Mohamed · Andriy Mnih -
2018 Poster: MetaGAN: An Adversarial Approach to Few-Shot Learning »
Ruixiang ZHANG · Tong Che · Zoubin Ghahramani · Yoshua Bengio · Yangqiu Song -
2017 : Panel Session »
Neil Lawrence · Finale Doshi-Velez · Zoubin Ghahramani · Yann LeCun · Max Welling · Yee Whye Teh · Ole Winther -
2017 Workshop: Bayesian Deep Learning »
Yarin Gal · José Miguel Hernández-Lobato · Christos Louizos · Andrew Wilson · Andrew Wilson · Diederik Kingma · Zoubin Ghahramani · Kevin Murphy · Max Welling -
2017 : Panel: On the Foundations and Future of Approximate Inference »
David Blei · Zoubin Ghahramani · Katherine Heller · Tim Salimans · Max Welling · Matthew D. Hoffman -
2017 : Panel: "Should we prioritize research on human-like AI or something different?" »
Cynthia Dwork · David Runciman · Zoubin Ghahramani -
2017 Symposium: Kinds of intelligence: types, tests and meeting the needs of society »
José Hernández-Orallo · Zoubin Ghahramani · Tomaso Poggio · Adrian Weller · Matthew Crosby -
2017 Poster: An inner-loop free solution to inverse problems using deep neural networks »
Kai Fan · Qi Wei · Lawrence Carin · Katherine Heller -
2017 Poster: Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning »
Shixiang (Shane) Gu · Timothy Lillicrap · Richard Turner · Zoubin Ghahramani · Bernhard Schölkopf · Sergey Levine -
2016 : Panel Discussion »
Shakir Mohamed · David Blei · Ryan Adams · José Miguel Hernández-Lobato · Ian Goodfellow · Yarin Gal -
2016 : Bayesian Agents: Bayesian Reasoning and Deep Learning in Agent-based Systems »
Shakir Mohamed -
2016 : Automatic Discovery of the Statistical Types of Variables in a Dataset »
Isabel Valera · Zoubin Ghahramani -
2016 : History of Bayesian neural networks »
Zoubin Ghahramani -
2016 Workshop: Bayesian Deep Learning »
Yarin Gal · Christos Louizos · Zoubin Ghahramani · Kevin Murphy · Max Welling -
2016 Workshop: Towards an Artificial Intelligence for Data Science »
Charles Sutton · James Geddes · Zoubin Ghahramani · Padhraic Smyth · Chris Williams -
2016 : How Machine Learning Research Can Address Key Societal and Governance Issues »
Zoubin Ghahramani -
2016 Workshop: People and machines: Public views on machine learning, and what this means for machine learning researchers »
Susannah Odell · Peter Donnelly · Jessica Montgomery · Sabine Hauert · Zoubin Ghahramani · Katherine Gorman -
2016 Poster: Unsupervised Learning of 3D Structure from Images »
Danilo Jimenez Rezende · S. M. Ali Eslami · Shakir Mohamed · Peter Battaglia · Max Jaderberg · Nicolas Heess -
2016 Poster: A Theoretically Grounded Application of Dropout in Recurrent Neural Networks »
Yarin Gal · Zoubin Ghahramani -
2016 Poster: Distributed Flexible Nonlinear Tensor Factorization »
Shandian Zhe · Kai Zhang · Pengyuan Wang · Kuang-chih Lee · Zenglin Xu · Yuan Qi · Zoubin Ghahramani -
2016 Tutorial: Variational Inference: Foundations and Modern Methods »
David Blei · Shakir Mohamed · Rajesh Ranganath -
2015 : Bayesian Optimization »
Zoubin Ghahramani · Bobak Shahriari -
2015 Workshop: Black box learning and inference »
Josh Tenenbaum · Jan-Willem van de Meent · Tejas Kulkarni · S. M. Ali Eslami · Brooks Paige · Frank Wood · Zoubin Ghahramani -
2015 Workshop: Advances in Approximate Bayesian Inference »
Dustin Tran · Tamara Broderick · Stephan Mandt · James McInerney · Shakir Mohamed · Alp Kucukelbir · Matthew D. Hoffman · Neil Lawrence · David Blei -
2015 Poster: Parallelizing MCMC with Random Partition Trees »
Xiangyu Wang · Fangjian Guo · Katherine Heller · David B Dunson -
2015 Poster: Particle Gibbs for Infinite Hidden Markov Models »
Nilesh Tripuraneni · Shixiang (Shane) Gu · Hong Ge · Zoubin Ghahramani -
2015 Poster: Neural Adaptive Sequential Monte Carlo »
Shixiang (Shane) Gu · Zoubin Ghahramani · Richard Turner -
2015 Poster: MCMC for Variationally Sparse Gaussian Processes »
James Hensman · Alexander Matthews · Maurizio Filippone · Zoubin Ghahramani -
2015 Poster: Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions »
Amar Shah · Zoubin Ghahramani -
2015 Invited Talk: Probabilistic Machine Learning: Foundations and Frontiers »
Zoubin Ghahramani -
2015 Poster: Statistical Model Criticism using Kernel Two Sample Tests »
James R Lloyd · Zoubin Ghahramani -
2015 Poster: Fast Second Order Stochastic Backpropagation for Variational Inference »
Kai Fan · Ziteng Wang · Jeff Beck · James Kwok · Katherine Heller -
2015 Poster: Variational Information Maximisation for Intrinsically Motivated Reinforcement Learning »
Shakir Mohamed · Danilo Jimenez Rezende -
2014 Workshop: Advances in Variational Inference »
David Blei · Shakir Mohamed · Michael Jordan · Charles Blundell · Tamara Broderick · Matthew D. Hoffman -
2014 Workshop: Bayesian Optimization in Academia and Industry »
Zoubin Ghahramani · Ryan Adams · Matthew Hoffman · Kevin Swersky · Jasper Snoek -
2014 Poster: Predictive Entropy Search for Efficient Global Optimization of Black-box Functions »
José Miguel Hernández-Lobato · Matthew Hoffman · Zoubin Ghahramani -
2014 Poster: Semi-supervised Learning with Deep Generative Models »
Diederik Kingma · Shakir Mohamed · Danilo Jimenez Rezende · Max Welling -
2014 Poster: Gaussian Process Volatility Model »
Yue Wu · José Miguel Hernández-Lobato · Zoubin Ghahramani -
2014 Spotlight: Semi-supervised Learning with Deep Generative Models »
Diederik Kingma · Shakir Mohamed · Danilo Jimenez Rezende · Max Welling -
2014 Spotlight: Predictive Entropy Search for Efficient Global Optimization of Black-box Functions »
José Miguel Hernández-Lobato · Matthew Hoffman · Zoubin Ghahramani -
2014 Poster: General Table Completion using a Bayesian Nonparametric Model »
Isabel Valera · Zoubin Ghahramani -
2013 Workshop: Probabilistic Models for Big Data »
Neil D Lawrence · Joaquin Quiñonero-Candela · Tianshi Gao · James Hensman · Zoubin Ghahramani · Max Welling · David Blei · Ralf Herbrich -
2013 Session: Oral Session 5 »
Zoubin Ghahramani -
2012 Workshop: Bayesian Optimization and Decision Making »
Javad Azimi · Roman Garnett · Frank R Hutter · Shakir Mohamed -
2012 Poster: Collaborative Gaussian Processes for Preference Learning »
Neil Houlsby · José Miguel Hernández-Lobato · Ferenc Huszar · Zoubin Ghahramani -
2012 Poster: Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models »
Jeff Beck · Katherine Heller · Alexandre Pouget -
2012 Spotlight: Complex Inference in Neural Circuits with Probabilistic Population Codes and Topic Models »
Jeff Beck · Katherine Heller · Alexandre Pouget -
2012 Poster: Modelling Reciprocating Relationships with Hawkes processes »
Charles Blundell · Katherine Heller · Jeff Beck -
2012 Poster: A nonparametric variable clustering model »
David A Knowles · Konstantina Palla · Zoubin Ghahramani -
2012 Session: Oral Session 7 »
Katherine Heller -
2012 Poster: Expectation Propagation in Gaussian Process Dynamical Systems »
Marc Deisenroth · Shakir Mohamed -
2012 Poster: Random function priors for exchangeable graphs and arrays »
James R Lloyd · Daniel Roy · Peter Orbanz · Zoubin Ghahramani -
2012 Poster: Active Learning of Model Evidence Using Bayesian Quadrature »
Michael A Osborne · David Duvenaud · Roman Garnett · Carl Edward Rasmussen · Stephen J Roberts · Zoubin Ghahramani -
2012 Poster: Fast Bayesian Inference for Non-Conjugate Gaussian Process Regression »
Mohammad Emtiyaz Khan · Shakir Mohamed · Kevin Murphy -
2012 Poster: Continuous Relaxations for Discrete Hamiltonian Monte Carlo »
Zoubin Ghahramani · Yichuan Zhang · Charles Sutton · Amos Storkey -
2012 Spotlight: Continuous Relaxations for Discrete Hamiltonian Monte Carlo »
Zoubin Ghahramani · Yichuan Zhang · Charles Sutton · Amos Storkey -
2012 Spotlight: Modelling Reciprocating Relationships with Hawkes processes »
Charles Blundell · Katherine Heller · Jeff Beck -
2011 Workshop: Copulas in Machine Learning »
Gal Elidan · Zoubin Ghahramani · John Lafferty -
2011 Poster: Testing a Bayesian Measure of Representativeness Using a Large Image Database »
Joshua T Abbott · Katherine Heller · Zoubin Ghahramani · Tom Griffiths -
2011 Session: Oral Session 7 »
Katherine Heller -
2010 Workshop: Transfer Learning Via Rich Generative Models. »
Russ Salakhutdinov · Ryan Adams · Josh Tenenbaum · Zoubin Ghahramani · Tom Griffiths -
2010 Talk: Unifying Views in Unsupervised Learning »
Zoubin Ghahramani -
2010 Oral: Tree-Structured Stick Breaking for Hierarchical Data »
Ryan Adams · Zoubin Ghahramani · Michael Jordan -
2010 Session: Spotlights Session 9 »
Katherine Heller -
2010 Session: Oral Session 11 »
Katherine Heller -
2010 Poster: Tree-Structured Stick Breaking for Hierarchical Data »
Ryan Adams · Zoubin Ghahramani · Michael Jordan -
2010 Spotlight: Copula Processes »
Andrew Wilson · Zoubin Ghahramani -
2010 Poster: Copula Processes »
Andrew Wilson · Zoubin Ghahramani -
2009 Workshop: Nonparametric Bayes »
Dilan Gorur · Francois Caron · Yee Whye Teh · David B Dunson · Zoubin Ghahramani · Michael Jordan -
2009 Workshop: Applications for Topic Models: Text and Beyond »
David Blei · Jordan Boyd-Graber · Jonathan Chang · Katherine Heller · Hanna Wallach -
2009 Poster: Hierarchical Learning of Dimensional Biases in Human Categorization »
Katherine Heller · Adam Sanborn · Nick Chater -
2009 Spotlight: Hierarchical Learning of Dimensional Biases in Human Categorization »
Katherine Heller · Adam Sanborn · Nick Chater -
2009 Poster: Large Scale Nonparametric Bayesian Inference: Data Parallelisation in the Indian Buffet Process »
Shakir Mohamed · David A Knowles · Zoubin Ghahramani · Finale P Doshi-Velez -
2008 Poster: The Infinite Factorial Hidden Markov Model »
Jurgen Van Gael · Yee Whye Teh · Zoubin Ghahramani -
2008 Spotlight: The Infinite Factorial Hidden Markov Model »
Jurgen Van Gael · Yee Whye Teh · Zoubin Ghahramani -
2007 Poster: Hidden Common Cause Relations in Relational Learning »
Ricardo Silva · Wei Chu · Zoubin Ghahramani -
2007 Spotlight: Hidden Common Cause Relations in Relational Learning »
Ricardo Silva · Wei Chu · Zoubin Ghahramani -
2006 Poster: Relational Learning with Gaussian Processes »
Wei Chu · Vikas Sindhwani · Zoubin Ghahramani · Sathiya Selvaraj Keerthi -
2006 Poster: Modeling Dyadic Data with Binary Latent Features »
Ted Meeds · Zoubin Ghahramani · Radford M Neal · Sam T Roweis -
2006 Spotlight: Modeling Dyadic Data with Binary Latent Features »
Ted Meeds · Zoubin Ghahramani · Radford M Neal · Sam T Roweis