Timezone: »
In this paper we describe how MAP inference can be used to sample efficiently from Gibbs distributions. Specifically, we provide means for drawing either approximate or unbiased samples from Gibbs' distributions by introducing low dimensional perturbations and solving the corresponding MAP assignments. Our approach also leads to new ways to derive lower bounds on partition functions. We demonstrate empirically that our method excels in the typical "high signal - high coupling'' regime. The setting results in ragged energy landscapes that are challenging for alternative approaches to sampling and/or lower bounds.
Author Information
Tamir Hazan (Technion)
Subhransu Maji (University of Massachusetts, Amherst)
Tommi Jaakkola (MIT)
Tommi Jaakkola is a professor of Electrical Engineering and Computer Science at MIT. He received an M.Sc. degree in theoretical physics from Helsinki University of Technology, and Ph.D. from MIT in computational neuroscience. Following a Sloan postdoctoral fellowship in computational molecular biology, he joined the MIT faculty in 1998. His research interests include statistical inference, graphical models, and large scale modern estimation problems with predominantly incomplete data.
More from the Same Authors
-
2020 Poster: Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies »
Itai Gat · Idan Schwartz · Alexander Schwing · Tamir Hazan -
2020 Poster: Direct Policy Gradients: Direct Optimization of Policies in Discrete Action Spaces »
Guy Lorberbom · Chris J. Maddison · Nicolas Heess · Tamir Hazan · Daniel Tarlow -
2019 Poster: Solving graph compression via optimal transport »
Vikas Garg · Tommi Jaakkola -
2019 Poster: Generative Models for Graph-Based Protein Design »
John Ingraham · Vikas Garg · Regina Barzilay · Tommi Jaakkola -
2019 Poster: Direct Optimization through $\arg \max$ for Discrete Variational Auto-Encoder »
Guy Lorberbom · Andreea Gane · Tommi Jaakkola · Tamir Hazan -
2019 Poster: Tight Certificates of Adversarial Robustness for Randomly Smoothed Classifiers »
Guang-He Lee · Yang Yuan · Shiyu Chang · Tommi Jaakkola -
2019 Poster: A Game Theoretic Approach to Class-wise Selective Rationalization »
Shiyu Chang · Yang Zhang · Mo Yu · Tommi Jaakkola -
2018 Poster: Towards Robust Interpretability with Self-Explaining Neural Networks »
David Alvarez-Melis · Tommi Jaakkola -
2017 Poster: Local Aggregative Games »
Vikas Garg · Tommi Jaakkola -
2017 Poster: Style Transfer from Non-Parallel Text by Cross-Alignment »
Tianxiao Shen · Tao Lei · Regina Barzilay · Tommi Jaakkola -
2017 Spotlight: Style Transfer from Non-parallel Text by Cross-Alignment »
Tianxiao Shen · Tao Lei · Regina Barzilay · Tommi Jaakkola -
2017 Poster: Predicting Organic Reaction Outcomes with Weisfeiler-Lehman Network »
Wengong Jin · Connor Coley · Regina Barzilay · Tommi Jaakkola -
2017 Poster: High-Order Attention Models for Visual Question Answering »
Idan Schwartz · Alexander Schwing · Tamir Hazan -
2016 Poster: Constraints Based Convex Belief Propagation »
Yaniv Tenzer · Alex Schwing · Kevin Gimpel · Tamir Hazan -
2016 Poster: Learning Tree Structured Potential Games »
Vikas Garg · Tommi Jaakkola -
2015 Poster: From random walks to distances on unweighted graphs »
Tatsunori Hashimoto · Yi Sun · Tommi Jaakkola -
2015 Poster: Principal Differences Analysis: Interpretable Characterization of Differences between Distributions »
Jonas Mueller · Tommi Jaakkola -
2014 Workshop: Perturbations, Optimization, and Statistics »
Tamir Hazan · George Papandreou · Daniel Tarlow -
2014 Poster: Controlling privacy in recommender systems »
Yu Xin · Tommi Jaakkola -
2013 Workshop: Perturbations, Optimization, and Statistics »
Tamir Hazan · George Papandreou · Sasha Rakhlin · Daniel Tarlow -
2013 Poster: Learning Efficient Random Maximum A-Posteriori Predictors with Non-Decomposable Loss Functions »
Tamir Hazan · Subhransu Maji · Joseph Keshet · Tommi Jaakkola -
2012 Workshop: Perturbations, Optimization, and Statistics »
Tamir Hazan · George Papandreou · Daniel Tarlow -
2012 Workshop: Machine Learning Approaches to Mobile Context Awareness »
Katherine Ellis · Gert Lanckriet · Tommi Jaakkola · Lenny Grokop -
2012 Poster: Globally Convergent Dual MAP LP Relaxation Solvers using Fenchel-Young Margins »
Alex Schwing · Tamir Hazan · Marc Pollefeys · Raquel Urtasun -
2012 Poster: Convergence Rate Analysis of MAP Coordinate Minimization Algorithms »
Ofer Meshi · Tommi Jaakkola · Amir Globerson -
2011 Tutorial: Linear Programming Relaxations for Graphical Models »
Amir Globerson · Tommi Jaakkola -
2010 Spotlight: More data means less inference: A pseudo-max approach to structured learning »
David Sontag · Ofer Meshi · Tommi Jaakkola · Amir Globerson -
2010 Poster: More data means less inference: A pseudo-max approach to structured learning »
David Sontag · Ofer Meshi · Tommi Jaakkola · Amir Globerson -
2010 Poster: A Primal-Dual Message-Passing Algorithm for Approximated Large Scale Structured Prediction »
Tamir Hazan · Raquel Urtasun -
2010 Poster: Direct Loss Minimization for Structured Prediction »
David A McAllester · Tamir Hazan · Joseph Keshet -
2008 Workshop: Approximate inference - how far have we come? »
Amir Globerson · David Sontag · Tommi Jaakkola -
2008 Poster: Clusters and Coarse Partitions in LP Relaxations »
David Sontag · Amir Globerson · Tommi Jaakkola -
2008 Spotlight: Clusters and Coarse Partitions in LP Relaxations »
David Sontag · Amir Globerson · Tommi Jaakkola -
2007 Oral: New Outer Bounds on the Marginal Polytope »
David Sontag · Tommi Jaakkola -
2007 Poster: New Outer Bounds on the Marginal Polytope »
David Sontag · Tommi Jaakkola -
2007 Poster: Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations »
Amir Globerson · Tommi Jaakkola -
2006 Talk: Approximate inference using planar graph decomposition »
Amir Globerson · Tommi Jaakkola -
2006 Poster: Approximate inference using planar graph decomposition »
Amir Globerson · Tommi Jaakkola -
2006 Poster: Game Theoretic Algorithms for Protein-DNA binding »
Luis Perez-Breva · Luis E Ortiz · Chen-Hsiang Yeang · Tommi Jaakkola -
2006 Spotlight: Game Theoretic Algorithms for Protein-DNA binding »
Luis Perez-Breva · Luis E Ortiz · Chen-Hsiang Yeang · Tommi Jaakkola -
2006 Poster: Parameter Expanded Variational Bayesian Methods »
Yuan (Alan) Qi · Tommi Jaakkola