Timezone: »
A structural equation model (SEM) is an effective framework to reason over causal relationships represented via a directed acyclic graph (DAG).Recent advances have enabled effective maximum-likelihood point estimation of DAGs from observational data. However, a point estimate may not accurately capture the uncertainty in inferring the underlying graph in practical scenarios, wherein the true DAG is non-identifiable and/or the observed dataset is limited.We propose Bayesian Causal Discovery Nets (BCD Nets), a variational inference framework for estimating a distribution over DAGs characterizing a linear-Gaussian SEM.Developing a full Bayesian posterior over DAGs is challenging due to the the discrete and combinatorial nature of graphs.We analyse key design choices for scalable VI over DAGs, such as 1) the parametrization of DAGs via an expressive variational family, 2) a continuous relaxation that enables low-variance stochastic optimization, and 3) suitable priors over the latent variables.We provide a series of experiments on real and synthetic data showing that BCD Nets outperform maximum-likelihood methods on standard causal discovery metrics such as structural Hamming distance in low data regimes.
Author Information
Chris Cundy (Stanford University)
Aditya Grover (University of California, Los Angeles)
Stefano Ermon (Stanford)
More from the Same Authors
-
2020 : Paper 46: Disagreement-Regularized Imitation of Complex Multi-Agent Interactions »
Jiaming Song · Stefano Ermon -
2021 Spotlight: IQ-Learn: Inverse soft-Q Learning for Imitation »
Divyansh Garg · Shuvam Chakraborty · Chris Cundy · Jiaming Song · Stefano Ermon -
2021 Spotlight: Maximum Likelihood Training of Score-Based Diffusion Models »
Yang Song · Conor Durkan · Iain Murray · Stefano Ermon -
2021 : SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning »
Christopher Yeh · Chenlin Meng · Sherrie Wang · Anne Driscoll · Erik Rozi · Patrick Liu · Jihyeon Lee · Marshall Burke · David Lobell · Stefano Ermon -
2021 : Scalable Variational Approaches for Bayesian Causal Discovery »
Chris Cundy · Aditya Grover · Stefano Ermon -
2021 : Likelihood-free Density Ratio Acquisition Functions are not Equivalent to Expected Improvements »
Jiaming Song · Stefano Ermon -
2022 : LMPriors: Pre-Trained Language Models as Task-Specific Priors »
Kristy Choi · Chris Cundy · Sanjari Srivastava · Stefano Ermon -
2022 : Relaxing the Kolmogorov Structure Function for Realistic Computational Constraints »
Yoonho Lee · Chelsea Finn · Stefano Ermon -
2022 : Regularizing Score-based Models with Score Fokker-Planck Equations »
Chieh-Hsin Lai · Yuhta Takida · Naoki Murata · Toshimitsu Uesaka · Yuki Mitsufuji · Stefano Ermon -
2022 : On Distillation of Guided Diffusion Models »
Chenlin Meng · Ruiqi Gao · Diederik Kingma · Stefano Ermon · Jonathan Ho · Tim Salimans -
2022 : JPEG Artifact Correction using Denoising Diffusion Restoration Models »
Bahjat Kawar · Jiaming Song · Stefano Ermon · Michael Elad -
2022 : But Are You Sure? Quantifying Uncertainty in Model Explanations »
Charles Marx · Youngsuk Park · Hilaf Hasson · Yuyang (Bernie) Wang · Stefano Ermon · Chaitanya Baru -
2023 Poster: UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild »
Can Qin · Shu Zhang · Ning Yu · Yihao Feng · Xinyi Yang · Yingbo Zhou · Huan Wang · Juan Carlos Niebles · Caiming Xiong · Silvio Savarese · Stefano Ermon · Yun Fu · Ran Xu -
2023 Poster: HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution »
Eric Nguyen · Michael Poli · Marjan Faizi · Armin Thomas · Michael Wornow · Callum Birch-Sykes · Stefano Massaroli · Aman Patel · Clayton Rabideau · Yoshua Bengio · Stefano Ermon · Christopher Ré · Stephen Baccus -
2023 Poster: Equivariant Flow Matching with Hybrid Probability Transport for 3D Molecule Generation »
Yuxuan Song · Jingjing Gong · Minkai Xu · Ziyao Cao · Yanyan Lan · Stefano Ermon · Hao Zhou · Wei-Ying Ma -
2023 Poster: Parallel Sampling of Diffusion Models »
Andy Shih · Suneel Belkhale · Stefano Ermon · Dorsa Sadigh · Nima Anari -
2023 Poster: Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions »
Stefano Massaroli · Michael Poli · Dan Fu · Hermann Kumbong · David Romero · Rom Parnichkun · Aman Timalsina · Quinn McIntyre · Beidi Chen · Atri Rudra · Ce Zhang · Christopher Ré · Stefano Ermon · Yoshua Bengio -
2023 Poster: Calibration by Distribution Matching: Trainable Kernel Calibration Metrics »
Charles Marx · Sofian Zalouk · Stefano Ermon -
2023 Poster: Direct Preference Optimization: Your Language Model is Secretly a Reward Model »
Rafael Rafailov · Archit Sharma · Eric Mitchell · Christopher D Manning · Stefano Ermon · Chelsea Finn -
2023 Poster: Scaling Riemannian Diffusion Models »
Aaron Lou · Minkai Xu · Adam Farris · Stefano Ermon -
2023 Poster: Holistic Evaluation of Text-to-Image Models »
Tony Lee · Michihiro Yasunaga · Chenlin Meng · Yifan Mai · Joon Sung Park · Agrim Gupta · Yunzhi Zhang · Deepak Narayanan · Hannah Teufel · Marco Bellagente · Minguk Kang · Taesung Park · Jure Leskovec · Jun-Yan Zhu · Fei-Fei Li · Jiajun Wu · Stefano Ermon · Percy Liang -
2023 Poster: GEO-Bench: Toward Foundation Models for Earth Monitoring »
Alexandre Lacoste · Nils Lehmann · Pau Rodriguez · Evan Sherwin · Hannah Kerner · Björn Lütjens · Jeremy Irvin · David Dao · Hamed Alemohammad · Alexandre Drouin · Mehmet Gunturkun · Gabriel Huang · David Vazquez · Dava Newman · Yoshua Bengio · Stefano Ermon · Xiaoxiang Zhu -
2023 Oral: Direct Preference Optimization: Your Language Model is Secretly a Reward Model »
Rafael Rafailov · Archit Sharma · Eric Mitchell · Christopher D Manning · Stefano Ermon · Chelsea Finn -
2023 Workshop: Adaptive Experimental Design and Active Learning in the Real World »
Willie Neiswanger · Mojmir Mutny · Ilija Bogunovic · Ava Soleimany · Zi Wang · Stefano Ermon · Andreas Krause -
2023 Workshop: Generative AI and Biology (GenBio@NeurIPS2023) »
Minkai Xu · Regina Barzilay · Jure Leskovec · Wenxian Shi · Menghua Wu · Zhenqiao Song · Lei Li · Fan Yang · Stefano Ermon -
2022 Poster: Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models »
Muyang Li · Ji Lin · Chenlin Meng · Stefano Ermon · Song Han · Jun-Yan Zhu -
2022 Poster: Concrete Score Matching: Generalized Score Matching for Discrete Data »
Chenlin Meng · Kristy Choi · Jiaming Song · Stefano Ermon -
2022 Poster: LISA: Learning Interpretable Skill Abstractions from Language »
Divyansh Garg · Skanda Vaidyanath · Kuno Kim · Jiaming Song · Stefano Ermon -
2022 Poster: Training and Inference on Any-Order Autoregressive Models the Right Way »
Andy Shih · Dorsa Sadigh · Stefano Ermon -
2022 Poster: SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery »
Yezhen Cong · Samar Khanna · Chenlin Meng · Patrick Liu · Erik Rozi · Yutong He · Marshall Burke · David Lobell · Stefano Ermon -
2022 Poster: FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness »
Tri Dao · Dan Fu · Stefano Ermon · Atri Rudra · Christopher Ré -
2022 Poster: Denoising Diffusion Restoration Models »
Bahjat Kawar · Michael Elad · Stefano Ermon · Jiaming Song -
2022 Poster: Generalizing Bayesian Optimization with Decision-theoretic Entropies »
Willie Neiswanger · Lantao Yu · Shengjia Zhao · Chenlin Meng · Stefano Ermon -
2022 Poster: Transform Once: Efficient Operator Learning in Frequency Domain »
Michael Poli · Stefano Massaroli · Federico Berto · Jinkyoo Park · Tri Dao · Christopher Ré · Stefano Ermon -
2022 Poster: Self-Similarity Priors: Neural Collages as Differentiable Fractal Representations »
Michael Poli · Winnie Xu · Stefano Massaroli · Chenlin Meng · Kuno Kim · Stefano Ermon -
2022 Poster: Improving Self-Supervised Learning by Characterizing Idealized Representations »
Yann Dubois · Stefano Ermon · Tatsunori Hashimoto · Percy Liang -
2022 Poster: Exploration via Planning for Information about the Optimal Trajectory »
Viraj Mehta · Ian Char · Joseph Abbate · Rory Conlin · Mark Boyer · Stefano Ermon · Jeff Schneider · Willie Neiswanger -
2021 : TorchDyn: Implicit Models and Neural Numerical Methods in PyTorch »
Michael Poli · Stefano Massaroli · Atsushi Yamashita · Hajime Asama · Jinkyoo Park · Stefano Ermon -
2021 : Cundy, Grover, Ermon - BCD Nets: Scalable Variational Approaches for Bayesian Causal Discovery »
Chris Cundy · Aditya Grover · Stefano Ermon -
2021 Oral: Moser Flow: Divergence-based Generative Modeling on Manifolds »
Noam Rozen · Aditya Grover · Maximilian Nickel · Yaron Lipman -
2021 Poster: HyperSPNs: Compact and Expressive Probabilistic Circuits »
Andy Shih · Dorsa Sadigh · Stefano Ermon -
2021 Poster: Imitation with Neural Density Models »
Kuno Kim · Akshat Jindal · Yang Song · Jiaming Song · Yanan Sui · Stefano Ermon -
2021 Poster: Reliable Decisions with Threshold Calibration »
Roshni Sahoo · Shengjia Zhao · Alyssa Chen · Stefano Ermon -
2021 Poster: D2C: Diffusion-Decoding Models for Few-Shot Conditional Generation »
Abhishek Sinha · Jiaming Song · Chenlin Meng · Stefano Ermon -
2021 Poster: Decision Transformer: Reinforcement Learning via Sequence Modeling »
Lili Chen · Kevin Lu · Aravind Rajeswaran · Kimin Lee · Aditya Grover · Misha Laskin · Pieter Abbeel · Aravind Srinivas · Igor Mordatch -
2021 Poster: Improving Compositionality of Neural Networks by Decoding Representations to Inputs »
Mike Wu · Noah Goodman · Stefano Ermon -
2021 Poster: Spatial-Temporal Super-Resolution of Satellite Imagery via Conditional Pixel Synthesis »
Yutong He · Dingjie Wang · Nicholas Lai · William Zhang · Chenlin Meng · Marshall Burke · David Lobell · Stefano Ermon -
2021 Poster: Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration »
Shengjia Zhao · Michael Kim · Roshni Sahoo · Tengyu Ma · Stefano Ermon -
2021 Poster: Estimating High Order Gradients of the Data Distribution by Denoising »
Chenlin Meng · Yang Song · Wenzhe Li · Stefano Ermon -
2021 Poster: Maximum Likelihood Training of Score-Based Diffusion Models »
Yang Song · Conor Durkan · Iain Murray · Stefano Ermon -
2021 Poster: Pseudo-Spherical Contrastive Divergence »
Lantao Yu · Jiaming Song · Yang Song · Stefano Ermon -
2021 Poster: IQ-Learn: Inverse soft-Q Learning for Imitation »
Divyansh Garg · Shuvam Chakraborty · Chris Cundy · Jiaming Song · Stefano Ermon -
2021 Poster: CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation »
Yusuke Tashiro · Jiaming Song · Yang Song · Stefano Ermon -
2021 Poster: PiRank: Scalable Learning To Rank via Differentiable Sorting »
Robin Swezey · Aditya Grover · Bruno Charron · Stefano Ermon -
2021 Poster: Moser Flow: Divergence-based Generative Modeling on Manifolds »
Noam Rozen · Aditya Grover · Maximilian Nickel · Yaron Lipman -
2020 : Contributed Talk: Reset-Free Lifelong Learning with Skill-Space Planning »
Kevin Lu · Aditya Grover · Pieter Abbeel · Igor Mordatch -
2020 : Stefano Emron - Generative Modeling via Denoising »
Stefano Ermon -
2020 Poster: Improved Techniques for Training Score-Based Generative Models »
Yang Song · Stefano Ermon -
2020 Poster: Probabilistic Circuits for Variational Inference in Discrete Graphical Models »
Andy Shih · Stefano Ermon -
2020 Poster: Efficient Learning of Generative Models via Finite-Difference Score Matching »
Tianyu Pang · Kun Xu · Chongxuan LI · Yang Song · Stefano Ermon · Jun Zhu -
2020 Poster: Belief Propagation Neural Networks »
Jonathan Kuck · Shuvam Chakraborty · Hao Tang · Rachel Luo · Jiaming Song · Ashish Sabharwal · Stefano Ermon -
2020 Poster: HiPPO: Recurrent Memory with Optimal Polynomial Projections »
Albert Gu · Tri Dao · Stefano Ermon · Atri Rudra · Christopher Ré -
2020 Spotlight: HiPPO: Recurrent Memory with Optimal Polynomial Projections »
Albert Gu · Tri Dao · Stefano Ermon · Atri Rudra · Christopher Ré -
2020 Poster: Autoregressive Score Matching »
Chenlin Meng · Lantao Yu · Yang Song · Jiaming Song · Stefano Ermon -
2020 Poster: Diversity can be Transferred: Output Diversification for White- and Black-box Attacks »
Yusuke Tashiro · Yang Song · Stefano Ermon -
2020 Poster: MOPO: Model-based Offline Policy Optimization »
Tianhe Yu · Garrett Thomas · Lantao Yu · Stefano Ermon · James Zou · Sergey Levine · Chelsea Finn · Tengyu Ma -
2020 Poster: Multi-label Contrastive Predictive Coding »
Jiaming Song · Stefano Ermon -
2020 Oral: Multi-label Contrastive Predictive Coding »
Jiaming Song · Stefano Ermon -
2019 : Poster session »
Jindong Gu · Alice Xiang · Atoosa Kasirzadeh · Zhiwei Han · Omar U. Florez · Frederik Harder · An-phi Nguyen · Amir Hossein Akhavan Rahnama · Michele Donini · Dylan Slack · Junaid Ali · Paramita Koley · Michiel Bakker · Anna Hilgard · Hailey James · Gonzalo Ramos · Jialin Lu · Jingying Yang · Margarita Boyarskaya · Martin Pawelczyk · Kacper Sokol · Mimansa Jaiswal · Umang Bhatt · David Alvarez-Melis · Aditya Grover · Charles Marx · Mengjiao (Sherry) Yang · Jingyan Wang · Gökhan Çapan · Hanchen Wang · Steffen Grünewälder · Moein Khajehnejad · Gourab Patro · Russell Kunes · Samuel Deng · Yuanting Liu · Luca Oneto · Mengze Li · Thomas Weber · Stefan Matthes · Duy Patrick Tu -
2019 : Poster Session »
Ethan Harris · Tom White · Oh Hyeon Choung · Takashi Shinozaki · Dipan Pal · Katherine L. Hermann · Judy Borowski · Camilo Fosco · Chaz Firestone · Vijay Veerabadran · Benjamin Lahner · Chaitanya Ryali · Fenil Doshi · Pulkit Singh · Sharon Zhou · Michel Besserve · Michael Chang · Anelise Newman · Mahesan Niranjan · Jonathon Hare · Daniela Mihai · Marios Savvides · Simon Kornblith · Christina M Funke · Aude Oliva · Virginia de Sa · Dmitry Krotov · Colin Conwell · George Alvarez · Alex Kolchinski · Shengjia Zhao · Mitchell Gordon · Michael Bernstein · Stefano Ermon · Arash Mehrjou · Bernhard Schölkopf · John Co-Reyes · Michael Janner · Jiajun Wu · Josh Tenenbaum · Sergey Levine · Yalda Mohsenzadeh · Zhenglong Zhou -
2019 Workshop: Information Theory and Machine Learning »
Shengjia Zhao · Jiaming Song · Yanjun Han · Kristy Choi · Pratyusha Kalluri · Ben Poole · Alex Dimakis · Jiantao Jiao · Tsachy Weissman · Stefano Ermon -
2019 Poster: Temporal FiLM: Capturing Long-Range Sequence Dependencies with Feature-Wise Modulations. »
Sawyer Birnbaum · Volodymyr Kuleshov · Zayd Enam · Pang Wei Koh · Stefano Ermon -
2019 Poster: MintNet: Building Invertible Neural Networks with Masked Convolutions »
Yang Song · Chenlin Meng · Stefano Ermon -
2019 Poster: Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting »
Aditya Grover · Jiaming Song · Ashish Kapoor · Kenneth Tran · Alekh Agarwal · Eric Horvitz · Stefano Ermon -
2019 Poster: Meta-Inverse Reinforcement Learning with Probabilistic Context Variables »
Lantao Yu · Tianhe Yu · Chelsea Finn · Stefano Ermon -
2019 Poster: Approximating the Permanent by Sampling from Adaptive Partitions »
Jonathan Kuck · Tri Dao · Hamid Rezatofighi · Ashish Sabharwal · Stefano Ermon -
2019 Poster: Generative Modeling by Estimating Gradients of the Data Distribution »
Yang Song · Stefano Ermon -
2019 Oral: Generative Modeling by Estimating Gradients of the Data Distribution »
Yang Song · Stefano Ermon -
2018 : Panel »
Paroma Varma · Aditya Grover · Will Hamilton · Jessica Hamrick · Thomas Kipf · Marinka Zitnik -
2018 Workshop: Relational Representation Learning »
Aditya Grover · Paroma Varma · Frederic Sala · Christopher Ré · Jennifer Neville · Stefano Ermon · Steven Holtzen -
2018 : Stefano Ermon (Stanford University): Weakly Supervised Spatio-temporal Regression »
Stefano Ermon -
2018 Poster: Streamlining Variational Inference for Constraint Satisfaction Problems »
Aditya Grover · Tudor Achim · Stefano Ermon -
2018 Poster: Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance »
Neal Jean · Sang Michael Xie · Stefano Ermon -
2018 Poster: Multi-Agent Generative Adversarial Imitation Learning »
Jiaming Song · Hongyu Ren · Dorsa Sadigh · Stefano Ermon -
2018 Poster: Constructing Unrestricted Adversarial Examples with Generative Models »
Yang Song · Rui Shu · Nate Kushman · Stefano Ermon -
2018 Poster: Bias and Generalization in Deep Generative Models: An Empirical Study »
Shengjia Zhao · Hongyu Ren · Arianna Yuan · Jiaming Song · Noah Goodman · Stefano Ermon -
2018 Spotlight: Bias and Generalization in Deep Generative Models: An Empirical Study »
Shengjia Zhao · Hongyu Ren · Arianna Yuan · Jiaming Song · Noah Goodman · Stefano Ermon -
2018 Poster: Amortized Inference Regularization »
Rui Shu · Hung Bui · Shengjia Zhao · Mykel J Kochenderfer · Stefano Ermon -
2017 : Generative Adversarial Imitation Learning, Stefano Ermon, Stanford »
Stefano Ermon -
2017 : Posters »
Reihaneh Rabbany · Tianxi Li · Jacob Carroll · Yin Cheng Ng · Xueyu Mao · Alexandre Hollocou · Jeric Briones · James Atwood · John Santerre · Natalie Klein · Pranamesh Chakraborty · Zahra Razaee · Chandan Singh · Arun Suggala · Beilun Wang · Andrew R. Lawrence · Aditya Grover · FARSHAD HARIRCHI · radhika arava · Qing Zhou · Takatomi Kubo · Josue Orellana · Govinda Kamath · Vivek Kumar Bagaria -
2017 : Stefano Ermon (Stanford): Measuring Progress Towards Sustainable Development Goals with Machine Learning »
Stefano Ermon -
2017 : Contributed talk 4: Graphite: Iterative Generative Modeling of Graphs »
Aditya Grover -
2017 Poster: A-NICE-MC: Adversarial Training for MCMC »
Jiaming Song · Shengjia Zhao · Stefano Ermon -
2017 Poster: InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations »
Yunzhu Li · Jiaming Song · Stefano Ermon -
2017 Poster: Neural Variational Inference and Learning in Undirected Graphical Models »
Volodymyr Kuleshov · Stefano Ermon -
2016 Poster: Solving Marginal MAP Problems with NP Oracles and Parity Constraints »
Yexiang Xue · zhiyuan li · Stefano Ermon · Carla Gomes · Bart Selman -
2016 Poster: Generative Adversarial Imitation Learning »
Jonathan Ho · Stefano Ermon -
2016 Poster: Variational Bayes on Monte Carlo Steroids »
Aditya Grover · Stefano Ermon -
2016 Poster: Adaptive Concentration Inequalities for Sequential Decision Problems »
Shengjia Zhao · Enze Zhou · Ashish Sabharwal · Stefano Ermon -
2013 Poster: Embed and Project: Discrete Sampling with Universal Hashing »
Stefano Ermon · Carla Gomes · Ashish Sabharwal · Bart Selman -
2012 Poster: Density Propagation and Improved Bounds on the Partition Function »
Stefano Ermon · Carla Gomes · Ashish Sabharwal · Bart Selman -
2011 Poster: Accelerated Adaptive Markov Chain for Partition Function Computation »
Stefano Ermon · Carla Gomes · Ashish Sabharwal · Bart Selman -
2011 Spotlight: Accelerated Adaptive Markov Chain for Partition Function Computation »
Stefano Ermon · Carla Gomes · Ashish Sabharwal · Bart Selman