EnergyBased Models (EBM) capture dependencies between variables by associating a scalar energy to each configuration of the variables. Given a set of observed variables, an EBM inference consists in finding configurations of unobserved variables that minimize the energy. Training an EBM consists in designing a loss function whose minimization will shape the energy surface so that desired variable configurations have lower energies than undesired configurations. EBM approaches have been applied with considerable success to such problems as natural language processing, biological sequence analysis, computer vision (object detection and recognition), image segmentation, image restoration, unsupervised feature learning, and dimensionality reduction.
The first part of the tutorial will introduce the concepts of energybased inference, will discuss the relationships with nonprobabilistic forms of graphical models (unnormalized factor graphs), and will give the conditions that the loss function must satisfy so that its minimization will cause the model to produce good decisions. The second part will discuss the relative merits of EBM approaches and probabilistic approaches. EBMs provide more flexibility than probabilistic approaches in the design of the energy function because of the absence of normalization. More importantly, when training complex probabilistic models, one is often faced with the problem of evaluating (or approximating) intractable sums or integrals. EBMs trained with appropriate loss functions sidestep this problem altogether. The third part will present several popular learning models in the light of the EBM framework. In particular, discriminative learning methods for "structured" outputs will be discussed including: discriminative HMMs, Graph Transformer Networks, Conditional Random Fields, Maximum Margin Markov Networks, and related approaches. A simple interpretation will be given for several approximate maximum likelihood methods such as products of experts models, variational bound methods, and Hinton's Contrastive Divergence. Lastly, a number of applications to vision, NLP and bioinformatics will be discussed.
Author Information
Yann LeCun (Facebook AI Research and New York University)
Yann LeCun is Director of AI Research at Facebook, and Silver Professor of Data Science, Computer Science, Neural Science, and Electrical Engineering at New York University. He received the Electrical Engineer Diploma from ESIEE, Paris in 1983, and a PhD in Computer Science from Université Pierre et Marie Curie (Paris) in 1987. After a postdoc at the University of Toronto, he joined AT&T Bell Laboratories in Holmdel, NJ in 1988. He became head of the Image Processing Research Department at AT&T LabsResearch in 1996, and joined NYU as a professor in 2003, after a brief period as a Fellow of the NEC Research Institute in Princeton. From 2012 to 2014 he directed NYU's initiative in data science and became the founding director of the NYU Center for Data Science. He was named Director of AI Research at Facebook in late 2013 and retains a parttime position on the NYU faculty. His current interests include AI, machine learning, computer perception, mobile robotics, and computational neuroscience. He has published over 180 technical papers and book chapters on these topics as well as on neural networks, handwriting recognition, image processing and compression, and on dedicated circuits for computer perception.
More from the Same Authors

2018 Poster: GLoMo: Unsupervised Learning of Transferable Relational Graphs »
Zhilin Yang · Jake Zhao · Bhuwan Dhingra · Kaiming He · William Cohen · Russ Salakhutdinov · Yann LeCun 
2017 Tutorial: Geometric Deep Learning on Graphs and Manifolds »
Michael Bronstein · Joan Bruna · arthur szlam · Xavier Bresson · Yann LeCun 
2016 Symposium: Deep Learning Symposium »
Yoshua Bengio · Yann LeCun · Navdeep Jaitly · Roger Grosse 
2016 Poster: Disentangling factors of variation in deep representation using adversarial training »
Michael Mathieu · Junbo Jake Zhao · Junbo (Jake) Zhao · Aditya Ramesh · Pablo Sprechmann · Yann LeCun 
2016 Invited Talk (Posner Lecture): Predictive Learning »
Yann LeCun 
2015 Poster: Learning to Linearize Under Uncertainty »
Ross Goroshin · Michael Mathieu · Yann LeCun 
2015 Poster: Characterlevel Convolutional Networks for Text Classification »
Xiang Zhang · Junbo (Jake) Zhao · Yann LeCun 
2015 Poster: Deep learning with Elastic Averaging SGD »
Sixin Zhang · Anna Choromanska · Yann LeCun 
2015 Spotlight: Deep learning with Elastic Averaging SGD »
Sixin Zhang · Anna Choromanska · Yann LeCun 
2015 Tutorial: Deep Learning »
Geoffrey E Hinton · Yoshua Bengio · Yann LeCun 
2014 Poster: Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation »
Emily Denton · Wojciech Zaremba · Joan Bruna · Yann LeCun · Rob Fergus 
2014 Poster: Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation »
Jonathan J Tompson · Arjun Jain · Yann LeCun · Christoph Bregler 
2013 Workshop: Neural Information Processing Scaled for Bioacoustics : NIPS4B »
Hervé GLOTIN · Yann LeCun · Thierry Artières · Stephane Mallat · Ofer Tchernichovski · Xanadu Halkias 
2011 Workshop: Deep Learning and Unsupervised Feature Learning »
Yoshua Bengio · Adam Coates · Yann LeCun · Nicolas Le Roux · Andrew Y Ng 
2011 Poster: Structured sparse coding via lateral inhibition »
arthur d szlam · Karol Gregor · Yann LeCun 
2010 Workshop: Deep Learning and Unsupervised Feature Learning »
Honglak Lee · Marc'Aurelio Ranzato · Yoshua Bengio · Geoffrey E Hinton · Yann LeCun · Andrew Y Ng 
2010 Demonstration: NeuFlow: a dataflow processor for convolutional nets and other realtime algorithms »
Yann LeCun 
2010 Spotlight: Learning Convolutional Feature Hierarchies for Visual Recognition »
koray kavukcuoglu · Pierre Sermanet · YLan Boureau · Karol Gregor · Michael Mathieu · Yann LeCun 
2010 Poster: Learning Convolutional Feature Hierarchies for Visual Recognition »
koray kavukcuoglu · Pierre Sermanet · YLan Boureau · Karol Gregor · Michael Mathieu · Yann LeCun 
2010 Poster: Regularized estimation of image statistics by Score Matching »
Durk Kingma · Yann LeCun 
2007 Poster: Sparse Feature Learning for Deep Belief Networks »
Marc'Aurelio Ranzato · YLan Boureau · Yann LeCun 
2006 Poster: Efficient Learning of Sparse Representations with an EnergyBased Model »
Marc'Aurelio Ranzato · Christopher Poultney · Sumit Chopra · Yann LeCun 
2006 Spotlight: Efficient Learning of Sparse Representations with an EnergyBased Model »
Marc'Aurelio Ranzato · Christopher Poultney · Sumit Chopra · Yann LeCun