Timezone: »
Poster
Learning Deep Disentangled Embeddings With the F-Statistic Loss
Karl Ridgeway · Michael Mozer
Deep-embedding methods aim to discover representations of a domain that make explicit the domain's class structure and thereby support few-shot learning. Disentangling methods aim to make explicit compositional or factorial structure. We combine these two active but independent lines of research and propose a new paradigm suitable for both goals. We propose and evaluate a novel loss function based on the $F$ statistic, which describes the separation of two or more distributions. By ensuring that distinct classes are well separated on a subset of embedding dimensions, we obtain embeddings that are useful for few-shot learning. By not requiring separation on all dimensions, we encourage the discovery of disentangled representations. Our embedding method matches or beats state-of-the-art, as evaluated by performance on recall@$k$ and few-shot learning tasks. Our method also obtains performance superior to a variety of alternatives on disentangling, as evaluated by two key properties of a disentangled representation: modularity and explicitness. The goal of our work is to obtain more interpretable, manipulable, and generalizable deep representations of concepts and categories.
Author Information
Karl Ridgeway (University of Colorado, Boulder)
Michael Mozer (Google Brain / U. Colorado)
More from the Same Authors
-
2018 Poster: Adapted Deep Embeddings: A Synthesis of Methods for k-Shot Inductive Transfer Learning »
Tyler Scott · Karl Ridgeway · Michael Mozer -
2018 Poster: Sparse Attentive Backtracking: Temporal Credit Assignment Through Reminding »
Nan Rosemary Ke · Anirudh Goyal · Olexa Bilaniuk · Jonathan Binas · Michael Mozer · Chris Pal · Yoshua Bengio -
2018 Spotlight: Sparse Attentive Backtracking: Temporal Credit Assignment Through Reminding »
Nan Rosemary Ke · Anirudh Goyal · Olexa Bilaniuk · Jonathan Binas · Michael Mozer · Chris Pal · Yoshua Bengio -
2018 Spotlight: Adapted Deep Embeddings: A Synthesis of Methods for k-Shot Inductive Transfer Learning »
Tyler Scott · Karl Ridgeway · Michael Mozer -
2017 : Access consciousness and the construction of actionable representations »
Michael C Mozer -
2017 : Workshop overview »
Michael Mozer · Angela Yu · Brenden Lake -
2017 Workshop: Cognitively Informed Artificial Intelligence: Insights From Natural Intelligence »
Michael Mozer · Brenden Lake · Angela Yu -
2016 : Overcoming temptation: Incentive design for intertemporal choice »
Michael Mozer -
2016 : Opening Remarks, Invited Talk: Michael C. Mozer »
Michael Mozer -
2014 Workshop: Human Propelled Machine Learning »
Richard Baraniuk · Michael Mozer · Divyanshu Vats · Christoph Studer · Andrew E Waters · Andrew Lan -
2014 Poster: Automatic Discovery of Cognitive Skills to Improve the Prediction of Student Learning »
Robert Lindsey · Mohammad Khajah · Michael Mozer -
2013 Poster: Optimizing Instructional Policies »
Robert Lindsey · Michael Mozer · William J Huggins · Harold Pashler -
2013 Oral: Optimizing Instructional Policies »
Robert Lindsey · Michael Mozer · William J Huggins · Harold Pashler -
2012 Workshop: Personalizing education with machine learning »
Michael Mozer · javier r movellan · Robert Lindsey · Jacob Whitehill -
2011 Poster: An Unsupervised Decontamination Procedure For Improving The Reliability Of Human Judgments »
Michael Mozer · Benjamin Link · Harold Pashler -
2010 Spotlight: Improving Human Judgments by Decontaminating Sequential Dependencies »
Michael Mozer · Harold Pashler · Matthew Wilder · Robert Lindsey · Matt Jones · Michael Jones -
2010 Poster: Improving Human Judgments by Decontaminating Sequential Dependencies »
Michael Mozer · Harold Pashler · Matthew Wilder · Robert Lindsey · Matt Jones · Michael Jones -
2009 Poster: Predicting the Optimal Spacing of Study: A Multiscale Context Model of Memory »
Michael Mozer · Harold Pashler · Nicholas Cepeda · Robert Lindsey · Edward Vul -
2009 Spotlight: Predicting the Optimal Spacing of Study: A Multiscale Context Model of Memory »
Michael Mozer · Harold Pashler · Nicholas Cepeda · Robert Lindsey · Edward Vul -
2009 Poster: Sequential effects reflect parallel learning of multiple environmental regularities »
Matthew Wilder · Matt Jones · Michael Mozer -
2008 Poster: Optimal Response Initiation: Why Recent Experience Matters »
Matt Jones · Michael Mozer · Sachiko Kinoshita -
2008 Spotlight: Optimal Response Initiation: Why Recent Experience Matters »
Matt Jones · Michael Mozer · Sachiko Kinoshita -
2008 Poster: Temporal Dynamics of Cognitive Control »
Jeremy Reynolds · Michael Mozer -
2007 Spotlight: Experience-Guided Search: A Theory of Attentional Control »
Michael Mozer · David Baldwin -
2007 Poster: Experience-Guided Search: A Theory of Attentional Control »
Michael Mozer · David Baldwin -
2006 Poster: Context Effects in Category Learning: An Investigation of Four Probabilistic Models »
Michael Mozer · Michael Jones · Michael Shettel