Timezone: »
Reinforcement learning and imitation learning are effective paradigms for learning controllers of dynamical systems from experience. These fields have been empowered by recent success in deep learning of differentiable parametric models, allowing end-to-end training of highly nonlinear controllers that encompass perception, memory, prediction, and decision making. The aptitude of these models to represent latent dynamics, high-level goals, and long-term outcomes is unfortunately curbed by the poor sample complexity of many current algorithms for learning these models from experience.
Probabilistic reinforcement learning and inference of control structure are emerging as promising approaches for avoiding prohibitive amounts of controller–system interactions. These methods leverage informative priors on useful behavior, as well as controller structure such as hierarchy and modularity, as useful inductive biases that reduce the effective size of policy search space and shape the optimization landscape. Intrinsic and self-supervised signals can further guide the training process of distinct internal components — such as perceptual embeddings, predictive models, exploration policies, and inter-agent communication — to break down the hard holistic problem of control into more efficiently learnable parts.
Effective inference methods are crucial for probabilistic approaches to reinforcement learning and structured control. Approximate control and model-free reinforcement learning exploit latent system structure and priors on policy structure, that are not directly evident in the controller–system interactions, and must be inferred by the learning algorithm. The growing interest of the reinforcement learning and optimal control community in the application of inference methods is synchronized well with the development by the probabilistic learning community of powerful inference techniques, such as probabilistic programming, variational inference, Gaussian processes, and nonparametric regression.
This workshop is a venue for the inference and reinforcement learning communities to come together in discussing recent advances, developing insights, and future potential in inference methods and their application to probabilistic reinforcement learning and structured control. The goal of this workshop is to catalyze tighter collaboration within and between the communities, that will be leveraged in upcoming years to rise to the challenges of real-world control problems.
=== Intel AI is proud to sponsor Infer2Control @ NeurIPS 2018 ===
Early detection of tumors. Predicting equipment failures before they happen. Having a natural conversation with your home or car. Making retail more personal than ever. This is Artificial Intelligence powered by Intel, and companies around the globe are using it to make money, save money, and advance the future of their industry. At Intel, we’re using decades of expertise in silicon, software, communications, memory and storage to create the new technologies that AI demands. Technologies that break barriers between data center and edge, server and network, training and inference, model and reality – maximizing the economics of AI to take data from theory to real-world success. Learn more: ai.intel.com
Sat 5:20 a.m. - 5:30 a.m.
|
Opening Remarks
(
Introduction
)
|
Roy Fox 🔗 |
Sat 5:30 a.m. - 6:00 a.m.
|
Control as Inference and Soft Deep RL (Sergey Levine)
(
Invited Talk
)
|
Sergey Levine 🔗 |
Sat 6:00 a.m. - 6:10 a.m.
|
Unsupervised Learning of Image Embedding for Continuous Control (Carlos Florensa)
(
Contributed Talk
)
|
Carlos Florensa 🔗 |
Sat 6:10 a.m. - 6:20 a.m.
|
Variational Inference Techniques for Sequential Decision Making in Generative Models (Igor Kiselev)
(
Contributed Talk
)
|
Igor Kiselev 🔗 |
Sat 6:20 a.m. - 6:30 a.m.
|
Probabilistic Planning with Sequential Monte Carlo (Alexandre Piché)
(
Contributed Talk
)
|
Alexandre Piche 🔗 |
Sat 6:30 a.m. - 7:00 a.m.
|
Inference and control of rules in human hierarchical reinforcement learning (Anne Collins)
(
Invited Talk
)
|
Anne Collins 🔗 |
Sat 7:00 a.m. - 7:30 a.m.
|
Hierarchical RL: From Prior Knowledge to Policies (Shie Mannor)
(
Invited Talk
)
|
Shie Mannor 🔗 |
Sat 7:30 a.m. - 8:00 a.m.
|
-- Coffee Break 1 --
|
🔗 |
Sat 8:00 a.m. - 8:30 a.m.
|
Off-policy Policy Optimization (Dale Schuurmans)
(
Invited Talk
)
|
Dale Schuurmans 🔗 |
Sat 8:30 a.m. - 8:45 a.m.
|
Spotlights 1
(
Spotlights
)
|
Ming-Xu Huang · Hao(Jackson) Cui · Arash Mehrjou · Yaqi Duan · Sharad Vikram · Angelina Wang · Karan Goel · Jonathan Hunt · Zhengwei Wu · Dinghan Shen · Mattie Fellows
|
Sat 8:45 a.m. - 9:15 a.m.
|
Poster Session 1
(
Poster Session
)
|
Kyle H Ambert · Brandon Araki · Xiya Cao · Sungjoon Choi · Hao(Jackson) Cui · Jonas Degrave · Yaqi Duan · Mattie Fellows · Carlos Florensa · Karan Goel · Aditya Gopalan · Ming-Xu Huang · Jonathan Hunt · Cyril Ibrahim · Brian Ichter · Maximilian Igl · Zheng Tracy Ke · Igor Kiselev · Anuj Mahajan · Arash Mehrjou · Karl Pertsch · Alexandre Piche · Nicholas Rhinehart · Thomas Ringstrom · Reazul Hasan Russel · Oleh Rybkin · Ion Stoica · Sharad Vikram · Angelina Wang · Ting-Han Wei · Abigail H Wen · I-Chen Wu · Zhengwei Wu · Linhai Xie · Dinghan Shen
|
Sat 9:15 a.m. - 10:45 a.m.
|
-- Lunch Break --
|
🔗 |
Sat 10:45 a.m. - 11:15 a.m.
|
Solving inference and control problems with the same machinery (Emo Todorov)
(
Invited Talk
)
|
Emo Todorov 🔗 |
Sat 11:15 a.m. - 11:30 a.m.
|
Spotlights 2
(
Spotlights
)
|
Aditya Gopalan · Sungjoon Choi · Thomas Ringstrom · Roy Fox · Jonas Degrave · Xiya Cao · Karl Pertsch · Maximilian Igl · Brian Ichter 🔗 |
Sat 11:30 a.m. - 12:00 p.m.
|
Inference and Control of Learning Behavior in Rodents (Ryan Adams)
(
Invited Talk
)
|
Ryan Adams 🔗 |
Sat 12:00 p.m. - 12:30 p.m.
|
-- Coffee Break 2 --
|
🔗 |
Sat 12:30 p.m. - 1:00 p.m.
|
On the Value of Knowing What You Don't Know: Learning to Sample and Sampling to Learn for Robot Planning (Leslie Kaelbling)
(
Invited Talk
)
|
Leslie Kaelbling 🔗 |
Sat 1:00 p.m. - 1:10 p.m.
|
Learning to Plan with Logical Automata (Brandon Araki)
(
Contributed Talk
)
|
Brandon Araki 🔗 |
Sat 1:10 p.m. - 1:20 p.m.
|
Tight Bayesian Ambiguity Sets for Robust MDPs (Reazul Hasan Russel)
(
Contributed Talk
)
|
Reazul Hasan Russel 🔗 |
Sat 1:20 p.m. - 1:30 p.m.
|
Deep Imitative Models for Flexible Inference, Planning, and Control (Nicholas Rhinehart)
(
Contributed Talk
)
|
Nicholas Rhinehart 🔗 |
Sat 1:30 p.m. - 2:00 p.m.
|
Probabilistic Reasoning for Reinforcement Learning (Nicolas Heess)
(
Invited Talk
)
|
Nicolas Heess 🔗 |
Sat 2:00 p.m. - 3:00 p.m.
|
Discussion Panel: Ryan Adams, Nicolas Heess, Leslie Kaelbling, Shie Mannor, Emo Todorov (moderator: Roy Fox)
(
Discussion Panel
)
|
Ryan Adams · Nicolas Heess · Leslie Kaelbling · Shie Mannor · Emo Todorov · Roy Fox 🔗 |
Sat 3:00 p.m. - 3:30 p.m.
|
Poster Session 2
(
Poster Session
)
|
🔗 |
Author Information
Leslie Kaelbling (MIT)
Martin Riedmiller (DeepMind)
Marc Toussaint (Universty Stuttgart)
Igor Mordatch (University of Washington)
Roy Fox (UC Berkeley)

[Roy Fox](royf.org) is an Assistant Professor and director of the Intelligent Dynamics Lab at the Department of Computer Science at UCI. His research interests include theory and applications of reinforcement learning, algorithmic game theory, information theory, and robotics. His current research focuses on structure, exploration, and optimization in deep reinforcement learning and imitation learning of virtual and physical agents and multi-agent systems. He was previously a postdoc at UC Berkeley, where he developed algorithms and systems that interact with humans to learn structured control policies for robotics and program synthesis.
Tuomas Haarnoja (UC Berkeley)
More from the Same Authors
-
2020 : Robotic gripper design with Evolutionary Strategies and Graph Element Networks »
Ferran Alet · Maria Bauza · Adarsh K Jeewajee · Max Thomsen · Alberto Rodriguez · Leslie Kaelbling · Tomás Lozano-Pérez -
2021 : The Neural MMO Platform for Massively Multiagent Research »
Joseph Suarez · Yilun Du · Clare Zhu · Igor Mordatch · Phillip Isola -
2021 : Brax - A Differentiable Physics Engine for Large Scale Rigid Body Simulation »
Daniel Freeman · Erik Frey · Anton Raichuk · Sertan Girgin · Igor Mordatch · Olivier Bachem -
2021 : Is Curiosity All You Need? On the Utility of Emergent Behaviours from Curious Exploration »
Oliver Groth · Markus Wulfmeier · Giulia Vezzani · Vibhavari Dasagi · Tim Hertweck · Roland Hafner · Nicolas Heess · Martin Riedmiller -
2021 : Temporal-Difference Value Estimation via Uncertainty-Guided Soft Updates »
Litian Liang · Yaosheng Xu · Stephen McAleer · Dailin Hu · Alexander Ihler · Pieter Abbeel · Roy Fox -
2021 : Target Entropy Annealing for Discrete Soft Actor-Critic »
Yaosheng Xu · Dailin Hu · Litian Liang · Stephen McAleer · Pieter Abbeel · Roy Fox -
2021 : Count-Based Temperature Scheduling for Maximum Entropy Reinforcement Learning »
Dailin Hu · Pieter Abbeel · Roy Fox -
2022 : Solving PDDL Planning Problems with Pretrained Large Language Models »
Tom Silver · Varun Hariprasad · Reece Shuttleworth · Nishanth Kumar · Tomás Lozano-Pérez · Leslie Kaelbling -
2022 : Fifteen-minute Competition Overview Video »
Nico Gürtler · Georg Martius · Pavel Kolev · Sebastian Blaes · Manuel Wuethrich · Markus Wulfmeier · Cansu Sancaktar · Martin Riedmiller · Arthur Allshire · Bernhard Schölkopf · Annika Buchholz · Stefan Bauer -
2022 : Feasible Adversarial Robust Reinforcement Learning for Underspecified Environments »
JB Lanier · Stephen McAleer · Pierre Baldi · Roy Fox -
2023 Poster: What Planning Problems Can A Relational Neural Network Solve? »
Jiayuan Mao · Tomás Lozano-Pérez · Josh Tenenbaum · Leslie Kaelbling -
2023 Poster: Hierarchical Planning with Foundation Models »
Anurag Ajay · Seungwook Han · Yilun Du · Shuang Li · Abhi Gupta · Tommi Jaakkola · Josh Tenenbaum · Leslie Kaelbling · Akash Srivastava · Pulkit Agrawal -
2022 Competition: Real Robot Challenge III - Learning Dexterous Manipulation from Offline Data in the Real World »
Nico Gürtler · Georg Martius · Sebastian Blaes · Pavel Kolev · Cansu Sancaktar · Stefan Bauer · Manuel Wuethrich · Markus Wulfmeier · Martin Riedmiller · Arthur Allshire · Annika Buchholz · Bernhard Schölkopf -
2022 : Panel Discussion »
Cynthia Rudin · Dan Bohus · Brenna Argall · Alison Gopnik · Igor Mordatch · Samuel Kaski -
2022 : Language models and interactive decision-making »
Igor Mordatch -
2022 Poster: PDSketch: Integrated Domain Programming, Learning, and Planning »
Jiayuan Mao · Tomás Lozano-Pérez · Josh Tenenbaum · Leslie Kaelbling -
2021 : Panel A: Deployable Learning Algorithms for Embodied Systems »
Shuran Song · Martin Riedmiller · Nick Roy · Aude G Billard · Angela Schoellig · SiQi Zhou -
2021 : Reinforcement Learning in Real-World Control Systems »
Martin Riedmiller -
2021 Poster: Unsupervised Learning of Compositional Energy Concepts »
Yilun Du · Shuang Li · Yash Sharma · Josh Tenenbaum · Igor Mordatch -
2021 Poster: Decision Transformer: Reinforcement Learning via Sequence Modeling »
Lili Chen · Kevin Lu · Aravind Rajeswaran · Kimin Lee · Aditya Grover · Misha Laskin · Pieter Abbeel · Aravind Srinivas · Igor Mordatch -
2021 Poster: Understanding End-to-End Model-Based Reinforcement Learning Methods as Implicit Parameterization »
Clement Gehring · Kenji Kawaguchi · Jiaoyang Huang · Leslie Kaelbling -
2021 Poster: Learning to Execute: Efficient Learning of Universal Plan-Conditioned Policies in Robotics »
Ingmar Schubert · Danny Driess · Ozgur S. Oguz · Marc Toussaint -
2021 Poster: XDO: A Double Oracle Algorithm for Extensive-Form Games »
Stephen McAleer · JB Lanier · Kevin A Wang · Pierre Baldi · Roy Fox -
2021 Poster: Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction time »
Ferran Alet · Maria Bauza · Kenji Kawaguchi · Nurullah Giray Kuru · Tomás Lozano-Pérez · Leslie Kaelbling -
2021 Poster: Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policies »
Tim Seyde · Igor Gilitschenski · Wilko Schwarting · Bartolomeo Stellato · Martin Riedmiller · Markus Wulfmeier · Daniela Rus -
2020 Poster: Adversarially-learned Inference via an Ensemble of Discrete Undirected Graphical Models »
Adarsh Keshav Jeewajee · Leslie Kaelbling -
2020 : Doing for our robots what nature did for us »
Leslie Kaelbling -
2019 : Poster and Coffee Break 2 »
Karol Hausman · Kefan Dong · Ken Goldberg · Lihong Li · Lin Yang · Lingxiao Wang · Lior Shani · Liwei Wang · Loren Amdahl-Culleton · Lucas Cassano · Marc Dymetman · Marc Bellemare · Marcin Tomczak · Margarita Castro · Marius Kloft · Marius-Constantin Dinu · Markus Holzleitner · Martha White · Mengdi Wang · Michael Jordan · Mihailo Jovanovic · Ming Yu · Minshuo Chen · Moonkyung Ryu · Muhammad Zaheer · Naman Agarwal · Nan Jiang · Niao He · Nikolaus Yasui · Nikos Karampatziakis · Nino Vieillard · Ofir Nachum · Olivier Pietquin · Ozan Sener · Pan Xu · Parameswaran Kamalaruban · Paul Mineiro · Paul Rolland · Philip Amortila · Pierre-Luc Bacon · Prakash Panangaden · Qi Cai · Qiang Liu · Quanquan Gu · Raihan Seraj · Richard Sutton · Rick Valenzano · Robert Dadashi · Rodrigo Toro Icarte · Roshan Shariff · Roy Fox · Ruosong Wang · Saeed Ghadimi · Samuel Sokota · Sean Sinclair · Sepp Hochreiter · Sergey Levine · Sergio Valcarcel Macua · Sham Kakade · Shangtong Zhang · Sheila McIlraith · Shie Mannor · Shimon Whiteson · Shuai Li · Shuang Qiu · Wai Lok Li · Siddhartha Banerjee · Sitao Luan · Tamer Basar · Thinh Doan · Tianhe Yu · Tianyi Liu · Tom Zahavy · Toryn Klassen · Tuo Zhao · Vicenç Gómez · Vincent Liu · Volkan Cevher · Wesley Suttle · Xiao-Wen Chang · Xiaohan Wei · Xiaotong Liu · Xingguo Li · Xinyi Chen · Xingyou Song · Yao Liu · YiDing Jiang · Yihao Feng · Yilun Du · Yinlam Chow · Yinyu Ye · Yishay Mansour · · Yonathan Efroni · Yongxin Chen · Yuanhao Wang · Bo Dai · Chen-Yu Wei · Harsh Shrivastava · Hongyang Zhang · Qinqing Zheng · SIDDHARTHA SATPATHI · Xueqing Liu · Andreu Vall -
2019 Poster: Neural Relational Inference with Fast Modular Meta-learning »
Ferran Alet · Erica Weng · Tomás Lozano-Pérez · Leslie Kaelbling -
2018 : Discussion Panel: Ryan Adams, Nicolas Heess, Leslie Kaelbling, Shie Mannor, Emo Todorov (moderator: Roy Fox) »
Ryan Adams · Nicolas Heess · Leslie Kaelbling · Shie Mannor · Emo Todorov · Roy Fox -
2018 : On the Value of Knowing What You Don't Know: Learning to Sample and Sampling to Learn for Robot Planning (Leslie Kaelbling) »
Leslie Kaelbling -
2018 : Spotlights 2 »
Aditya Gopalan · Sungjoon Choi · Thomas Ringstrom · Roy Fox · Jonas Degrave · Xiya Cao · Karl Pertsch · Maximilian Igl · Brian Ichter -
2018 : Leslie Kaelbling »
Leslie Kaelbling -
2018 : Opening Remarks »
Roy Fox -
2018 : Talk 9: Marc Toussaint - Models & Abstractions for Physical Reasoning »
Marc Toussaint -
2018 : Talk 8: Leslie Kaelbling - Learning models of very large hybrid domains »
Leslie Kaelbling -
2018 Workshop: Modeling the Physical World: Learning, Perception, and Control »
Jiajun Wu · Kelsey Allen · Kevin Smith · Jessica Hamrick · Emmanuel Dupoux · Marc Toussaint · Josh Tenenbaum -
2018 Poster: Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior »
Zi Wang · Beomjoon Kim · Leslie Kaelbling -
2018 Spotlight: Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior »
Zi Wang · Beomjoon Kim · Leslie Kaelbling -
2017 : Panel Discussion »
Matt Botvinick · Emma Brunskill · Marcos Campos · Jan Peters · Doina Precup · David Silver · Josh Tenenbaum · Roy Fox -
2017 : Opening Remarks »
Roy Fox -
2017 Workshop: Hierarchical Reinforcement Learning »
Andrew G Barto · Doina Precup · Shie Mannor · Tom Schaul · Roy Fox · Carlos Florensa -
2017 Workshop: Acting and Interacting in the Real World: Challenges in Robot Learning »
Ingmar Posner · Raia Hadsell · Martin Riedmiller · Markus Wulfmeier · Rohan Paul -
2016 : Marc Toussaint - Bayesian Optimization: Applications in Robotics and Better Hyperparameters »
Marc Toussaint -
2016 Poster: Backprop KF: Learning Discriminative Deterministic State Estimators »
Tuomas Haarnoja · Anurag Ajay · Sergey Levine · Pieter Abbeel -
2015 Poster: Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images »
Manuel Watter · Jost Springenberg · Joschka Boedecker · Martin Riedmiller -
2015 Poster: Interactive Control of Diverse Complex Characters with Neural Networks »
Igor Mordatch · Kendall Lowrey · Galen Andrew · Zoran Popovic · Emanuel Todorov -
2015 Oral: Interactive Control of Diverse Complex Characters with Neural Networks »
Igor Mordatch · Kendall Lowrey · Galen Andrew · Zoran Popovic · Emanuel Todorov -
2015 Poster: Bayesian Optimization with Exponential Convergence »
Kenji Kawaguchi · Leslie Kaelbling · Tomás Lozano-Pérez -
2014 Workshop: Autonomously Learning Robots »
Gerhard Neumann · Joelle Pineau · Peter Auer · Marc Toussaint -
2008 Poster: Multi-Agent Filtering with Infinitely Nested Beliefs »
Luke Zettlemoyer · Brian Milch · Leslie Kaelbling -
2008 Spotlight: Multi-Agent Filtering with Infinitely Nested Beliefs »
Luke Zettlemoyer · Brian Milch · Leslie Kaelbling -
2007 Workshop: The Grammar of Vision: Probabilistic Grammar-Based Models for Visual Scene Understanding and Object Categorization »
Virginia Savova · Josh Tenenbaum · Leslie Kaelbling · Alan Yuille