Timezone: »
Computing expected predictions of discriminative models is a fundamental task in machine learning that appears in many interesting applications such as fairness, handling missing values, and data analysis. Unfortunately, computing expectations of a discriminative model with respect to a probability distribution defined by an arbitrary generative model has been proven to be hard in general. In fact, the task is intractable even for simple models such as logistic regression and a naive Bayes distribution. In this paper, we identify a pair of generative and discriminative models that enables tractable computation of expectations, as well as moments of any order, of the latter with respect to the former in case of regression. Specifically, we consider expressive probabilistic circuits with certain structural constraints that support tractable probabilistic inference. Moreover, we exploit the tractable computation of high-order moments to derive an algorithm to approximate the expectations for classification scenarios in which exact computations are intractable. Our framework to compute expected predictions allows for handling of missing data during prediction time in a principled and accurate way and enables reasoning about the behavior of discriminative models. We empirically show our algorithm to consistently outperform standard imputation techniques on a variety of datasets. Finally, we illustrate how our framework can be used for exploratory data analysis.
Author Information
Pasha Khosravi (UCLA)
YooJung Choi (UCLA)
Yitao Liang (UCLA)
Antonio Vergari (University of California, Los Angeles)
Guy Van den Broeck (UCLA)
I am an Assistant Professor and Samueli Fellow at UCLA, in the Computer Science Department, where I direct the Statistical and Relational Artificial Intelligence (StarAI) lab. My research interests are in Machine Learning (Statistical Relational Learning, Tractable Learning), Knowledge Representation and Reasoning (Graphical Models, Lifted Probabilistic Inference, Knowledge Compilation), Applications of Probabilistic Reasoning and Learning (Probabilistic Programming, Probabilistic Databases), and Artificial Intelligence in general.
More from the Same Authors
-
2021 Spotlight: Tractable Regularization of Probabilistic Circuits »
Anji Liu · Guy Van den Broeck -
2022 : Panel Discussion: "Heading for a Unifying View on nCSI" »
Tobias Gerstenberg · Sriraam Natarajan · - Mausam · Guy Van den Broeck · Devendra Dhami -
2022 : AI can learn from data. But can it learn to reason? »
Guy Van den Broeck -
2022 : Panel »
Guy Van den Broeck · Cassio de Campos · Denis Maua · Kristian Kersting · Rianne van den Berg -
2022 : Q & A »
Antonio Vergari · YooJung Choi · Robert Peharz -
2022 Tutorial: Probabilistic Circuits: Representations, Inference, Learning and Applications »
Antonio Vergari · YooJung Choi · Robert Peharz -
2022 : Tutorial part 1 »
Antonio Vergari · YooJung Choi · Robert Peharz -
2022 : Generating High Fidelity Synthetic Data via Coreset selection and Entropic Regularization »
Omead Pooladzandi · Pasha Khosravi · Erik Nijkamp · Baharan Mirzasoleiman -
2022 Poster: Semantic Probabilistic Layers for Neuro-Symbolic Learning »
Kareem Ahmed · Stefano Teso · Kai-Wei Chang · Guy Van den Broeck · Antonio Vergari -
2022 Poster: Sparse Probabilistic Circuits via Pruning and Growing »
Meihua Dang · Anji Liu · Guy Van den Broeck -
2021 Workshop: Advances in Programming Languages and Neurosymbolic Systems (AIPLANS) »
Breandan Considine · Disha Shrivastava · David Yu-Tung Hui · Chin-Wei Huang · Shawn Tan · Xujie Si · Prakash Panangaden · Guy Van den Broeck · Daniel Tarlow -
2021 : AI workloads inside databases »
Guy Van den Broeck · Alexander Ratner · Benjamin Moseley · Konstantinos Karanasos · Parisa Kordjamshidi · Molham Aref · Arun Kumar -
2021 Poster: A Compositional Atlas of Tractable Circuit Operations for Probabilistic Inference »
Antonio Vergari · YooJung Choi · Anji Liu · Stefano Teso · Guy Van den Broeck -
2021 : PYLON: A PyTorch Framework for Learning with Constraints »
Kareem Ahmed · Tao Li · Nu Mai Thy Ton · Quan Guo · Kai-Wei Chang · Parisa Kordjamshidi · Vivek Srikumar · Guy Van den Broeck · Sameer Singh -
2021 Oral: A Compositional Atlas of Tractable Circuit Operations for Probabilistic Inference »
Antonio Vergari · YooJung Choi · Anji Liu · Stefano Teso · Guy Van den Broeck -
2021 Poster: Tractable Regularization of Probabilistic Circuits »
Anji Liu · Guy Van den Broeck -
2020 : Contributed talks 6: Group Fairness by Probabilistic Modeling with Latent Fair Decisions »
YooJung Choi · Guy Van den Broeck -
2020 Poster: Probabilistic Inference with Algebraic Constraints: Theoretical Limits and Practical Approximations »
Zhe Zeng · Paolo Morettin · Fanqi Yan · Antonio Vergari · Guy Van den Broeck -
2020 Spotlight: Probabilistic Inference with Algebraic Constraints: Theoretical Limits and Practical Approximations »
Zhe Zeng · Paolo Morettin · Fanqi Yan · Antonio Vergari · Guy Van den Broeck -
2020 Poster: Counterexample-Guided Learning of Monotonic Neural Networks »
Aishwarya Sivaraman · Golnoosh Farnadi · Todd Millstein · Guy Van den Broeck -
2019 : Break / Poster Session 1 »
Antonia Marcu · Yao-Yuan Yang · Pascale Gourdeau · Chen Zhu · Thodoris Lykouris · Jianfeng Chi · Mark Kozdoba · Arjun Nitin Bhagoji · Xiaoxia Wu · Jay Nandy · Michael T Smith · Bingyang Wen · Yuege Xie · Konstantinos Pitas · Suprosanna Shit · Maksym Andriushchenko · Dingli Yu · GaĆ«l Letarte · Misha Khodak · Hussein Mozannar · Chara Podimata · James Foulds · Yizhen Wang · Huishuai Zhang · Ondrej Kuzelka · Alexander Levine · Nan Lu · Zakaria Mhammedi · Paul Viallard · Diana Cai · Lovedeep Gondara · James Lucas · Yasaman Mahdaviyeh · Aristide Baratin · Rishi Bommasani · Alessandro Barp · Andrew Ilyas · Kaiwen Wu · Jens Behrmann · Omar Rivasplata · Amir Nazemi · Aditi Raghunathan · Will Stephenson · Sahil Singla · Akhil Gupta · YooJung Choi · Yannic Kilcher · Clare Lyle · Edoardo Manino · Andrew Bennett · Zhi Xu · Niladri Chatterji · Emre Barut · Flavien Prost · Rodrigo Toro Icarte · Arno Blaas · Chulhee Yun · Sahin Lale · YiDing Jiang · Tharun Kumar Reddy Medini · Ashkan Rezaei · Alexander Meinke · Stephen Mell · Gary Kazantsev · Shivam Garg · Aradhana Sinha · Vishnu Lokhande · Geovani Rizk · Han Zhao · Aditya Kumar Akash · Jikai Hou · Ali Ghodsi · Matthias Hein · Tyler Sypherd · Yichen Yang · Anastasia Pentina · Pierre Gillot · Antoine Ledent · Guy Gur-Ari · Noah MacAulay · Tianzong Zhang -
2019 : Invited Talk (Guy Van den Broeck) »
Guy Van den Broeck -
2019 : Poster Spotlights A (23 posters) »
DongHa Bahn · Xiaoran Xu · Shih-Chieh Su · Daniel Cunnington · Wonseok Hwang · Sarthak Dash · Alberto Camacho · Theodoros Salonidis · Shiyang Li · Yuyu Zhang · Habibeh Naderi · Zhe Zeng · Pasha Khosravi · Pedro Colon-Hernandez · Dimitris Diochnos · David Windridge · Robin Manhaeve · Vaishak Belle · Brendan Juba · Naveen Sundar Govindarajulu · Joe Bockhorst -
2019 Poster: Towards Hardware-Aware Tractable Learning of Probabilistic Models »
Laura Galindez Olascoaga · Wannes Meert · Nimish Shah · Marian Verhelst · Guy Van den Broeck -
2019 Poster: Smoothing Structured Decomposable Circuits »
Andy Shih · Guy Van den Broeck · Paul Beame · Antoine Amarilli -
2019 Spotlight: Smoothing Structured Decomposable Circuits »
Andy Shih · Guy Van den Broeck · Paul Beame · Antoine Amarilli -
2018 Poster: Approximate Knowledge Compilation by Online Collapsed Importance Sampling »
Tal Friedman · Guy Van den Broeck -
2018 Oral: Approximate Knowledge Compilation by Online Collapsed Importance Sampling »
Tal Friedman · Guy Van den Broeck -
2018 Poster: Inference Aided Reinforcement Learning for Incentive Mechanism Design in Crowdsourcing »
Zehong Hu · Yitao Liang · Jie Zhang · Zhao Li · Yang Liu -
2017 Workshop: NIPS Highlights (MLTrain), Learn How to code a paper with state of the art frameworks »
Alex Dimakis · Nikolaos Vasiloglou · Guy Van den Broeck · Alexander Ihler · Assaf Araki -
2016 Poster: New Liftable Classes for First-Order Probabilistic Inference »
Seyed Mehran Kazemi · Angelika Kimmig · Guy Van den Broeck · David Poole -
2015 Poster: Tractable Learning for Complex Probability Queries »
Jessa Bekker · Jesse Davis · Arthur Choi · Adnan Darwiche · Guy Van den Broeck -
2013 Poster: On the Complexity and Approximation of Binary Evidence in Lifted Inference »
Guy Van den Broeck · Adnan Darwiche -
2013 Spotlight: On the Complexity and Approximation of Binary Evidence in Lifted Inference »
Guy Van den Broeck · Adnan Darwiche -
2011 Poster: On the Completeness of First-Order Knowledge Compilation for Lifted Probabilistic Inference »
Guy Van den Broeck -
2011 Oral: On the Completeness of First-Order Knowledge Compilation for Lifted Probabilistic Inference »
Guy Van den Broeck