This is the public, feature-limited version of the conference webpage. After Registration and login please visit the full version.

Workshop: Offline Reinforcement Learning

Aviral Kumar, Rishabh Agarwal, George Tucker, Lihong Li, Doina Precup, Aviral Kumar

2020-12-12T09:00:00-08:00 - 2020-12-12T18:00:00-08:00
Abstract: The common paradigm in reinforcement learning (RL) assumes that an agent frequently interacts with the environment and learns using its own collected experience. This mode of operation is prohibitive for many complex real-world problems, where repeatedly collecting diverse data is expensive (e.g., robotics or educational agents) and/or dangerous (e.g., healthcare). Alternatively, Offline RL focuses on training agents with logged data in an offline fashion with no further environment interaction. Offline RL promises to bring forward a data-driven RL paradigm and carries the potential to scale up end-to-end learning approaches to real-world decision making tasks such as robotics, recommendation systems, dialogue generation, autonomous driving, healthcare systems and safety-critical applications. Recently, successful deep RL algorithms have been adapted to the offline RL setting and demonstrated a potential for success in a number of domains, however, significant algorithmic and practical challenges remain to be addressed. The goal of this workshop is to bring attention to offline RL, both from within and from outside the RL community discuss algorithmic challenges that need to be addressed, discuss potential real-world applications, discuss limitations and challenges, and come up with concrete problem statements and evaluation protocols, inspired from real-world applications, for the research community to work on.

For details on submission please visit: (Submission deadline: October 9, 11:59 pm PT)

Emma Brunskill (Stanford)
Finale Doshi-Velez (Harvard)
John Langford (Microsoft Research)
Nan Jiang (UIUC)
Brandyn White (Waymo Research)
Nando de Freitas (DeepMind)


To ask questions please use rocketchat, available only upon registration and login.


2020-12-12T08:50:00-08:00 - 2020-12-12T09:00:00-08:00
Aviral Kumar, George Tucker, Rishabh Agarwal
2020-12-12T09:00:00-08:00 - 2020-12-12T09:30:00-08:00
Offline RL
Nando de Freitas
2020-12-12T09:30:00-08:00 - 2020-12-12T09:40:00-08:00
Q&A w/ Nando de Freitas
2020-12-12T09:40:00-08:00 - 2020-12-12T09:50:00-08:00
Contributed Talk 1: Offline Reinforcement Learning by Solving Derived Non-Parametric MDPs
Aayam Shrestha
2020-12-12T09:50:00-08:00 - 2020-12-12T10:00:00-08:00
Contributed Talk 2: Chaining Behaviors from Data with Model-Free Reinforcement Learning
Avi Singh
2020-12-12T10:00:00-08:00 - 2020-12-12T10:10:00-08:00
Contributed Talk 3: Addressing Distribution Shift in Online Reinforcement Learning with Offline Datasets
Seunghyun Lee, Younggyo Seo, Kimin Lee
2020-12-12T10:10:00-08:00 - 2020-12-12T10:20:00-08:00
Contributed Talk 4: Addressing Extrapolation Error in Deep Offline Reinforcement Learning
Caglar Gulcehre
2020-12-12T10:20:00-08:00 - 2020-12-12T10:30:00-08:00
Q/A for Contributed Talks 1
2020-12-12T10:30:00-08:00 - 2020-12-12T11:20:00-08:00
Poster Session 1 (
2020-12-12T11:20:00-08:00 - 2020-12-12T11:50:00-08:00
Causal Structure Discovery in RL
John Langford
2020-12-12T11:50:00-08:00 - 2020-12-12T12:00:00-08:00
Q&A w/ John Langford
2020-12-12T12:00:00-08:00 - 2020-12-12T13:00:00-08:00
Emma Brunskill, Nan Jiang, Nando de Freitas, Finale Doshi-Velez, Sergey Levine, John Langford, Lihong Li, George Tucker, Rishabh Agarwal, Aviral Kumar
2020-12-12T13:10:00-08:00 - 2020-12-12T13:40:00-08:00
Learning a Multi-Agent Simulator from Offline Demonstrations
Brandyn White, Brandyn White
2020-12-12T13:40:00-08:00 - 2020-12-12T13:50:00-08:00
Q&A w/ Brandyn White
2020-12-12T13:50:00-08:00 - 2020-12-12T14:20:00-08:00
Towards Reliable Validation and Evaluation for Offline RL
Nan Jiang
2020-12-12T14:20:00-08:00 - 2020-12-12T14:30:00-08:00
Q&A w/ Nan Jiang
2020-12-12T14:30:00-08:00 - 2020-12-12T14:40:00-08:00
Contributed Talk 5: Latent Action Space for Offline Reinforcement Learning
Wenxuan Zhou
2020-12-12T14:40:00-08:00 - 2020-12-12T14:50:00-08:00
Contributed Talk 6: What are the Statistical Limits for Batch RL with Linear Function Approximation?
Ruosong Wang
2020-12-12T14:50:00-08:00 - 2020-12-12T15:00:00-08:00
Contributed Talk 7: Distilled Thompson Sampling: Practical and Efficient Thompson Sampling via Imitation Learning
Hong Namkoong
2020-12-12T15:00:00-08:00 - 2020-12-12T15:10:00-08:00
Contributed Talk 8: Batch-Constrained Distributional Reinforcement Learning for Session-based Recommendation
Diksha Garg
2020-12-12T15:15:00-08:00 - 2020-12-12T16:30:00-08:00
Poster Session 2 (
2020-12-12T16:30:00-08:00 - 2020-12-12T17:00:00-08:00
Counterfactuals and Offline RL
Emma Brunskill
2020-12-12T17:00:00-08:00 - 2020-12-12T17:10:00-08:00
Q&A w/ Emma Brunskill
2020-12-12T17:10:00-08:00 - 2020-12-12T17:40:00-08:00
Batch RL Models Built for Validation
Finale Doshi-Velez
2020-12-12T17:40:00-08:00 - 2020-12-12T17:50:00-08:00
Q&A w/ Finale Doshi-Velez
2020-12-12T17:50:00-08:00 - 2020-12-12T18:00:00-08:00
Closing Remarks