Timezone: »

 
Workshop
Reinforcement Learning under Partial Observability
Joni Pajarinen · Chris Amato · Pascal Poupart · David Hsu

Sat Dec 08 05:00 AM -- 03:30 PM (PST) @ Room 517 C
Event URL: https://sites.google.com/site/rlponips2018/ »

Reinforcement learning (RL) has succeeded in many challenging tasks such as Atari, Go, and Chess and even in high dimensional continuous domains such as robotics. Most impressive successes are in tasks where the agent observes the task features fully. However, in real world problems, the agent usually can only rely on partial observations. In real time games the agent makes only local observations; in robotics the agent has to cope with noisy sensors, occlusions, and unknown dynamics. Even more fundamentally, any agent without a full a priori world model or without full access to the system state, has to make decisions based on partial knowledge about the environment and its dynamics.

Reinforcement learning under partial observability has been tackled in the operations research, control, planning, and machine learning communities. One of the goals of the workshop is to bring researchers from different backgrounds together. Moreover, the workshop aims to highlight future applications. In addition to robotics where partial observability is a well known challenge, many diverse applications such as wireless networking, human-robot interaction and autonomous driving require taking partial observability into account.

Partial observability introduces unique challenges: the agent has to remember the past but also connect the present with potential futures requiring memory, exploration, and value propagation techniques that can handle partial observability. Current model-based methods can handle discrete values and take long term information gathering into account while model-free methods can handle high-dimensional continuous problems but often assume that the state space has been created for the problem at hand such that there is sufficient information for optimal decision making or just add memory to the policy without taking partial observability explicitly into account.

In this workshop, we want to go further and ask among others the following questions.
* How can we extend deep RL methods to robustly solve partially observable problems?
* Can we learn concise abstractions of history that are sufficient for high-quality decision-making?
* There have been several successes in decision making under partial observability despite the inherent challenges. Can we characterize problems where computing good policies is feasible?
* Since decision making is hard under partial observability do we want to use more complex models and solve them approximately or use (inaccurate) simple models and solve them exactly? Or not use models at all?
* How can we use control theory together with reinforcement learning to advance decision making under partial observability?
* Can we combine the strengths of model-based and model-free methods under partial observability?
* Can recent method improvements in general RL already tackle some partially observable applications which were not previously possible?
* How do we scale up reinforcement learning in multi-agent systems with partial observability?
* Do hierarchical models / temporal abstraction improve RL efficiency under partial observability?

Sat 5:30 a.m. - 5:40 a.m. [iCal]
Opening Remarks
Sat 5:40 a.m. - 6:05 a.m. [iCal]
Joelle Pineau (Talk)
Joelle Pineau
Sat 6:05 a.m. - 6:30 a.m. [iCal]
Leslie Kaelbling (Talk)
Leslie Kaelbling
Sat 6:30 a.m. - 6:45 a.m. [iCal]
Contributed Talk 1: High-Level Strategy Selection under Partial Observability in StarCraft: Brood War (Talk)
Sat 6:45 a.m. - 7:10 a.m. [iCal]
David Silver (Talk)
David Silver
Sat 7:10 a.m. - 7:25 a.m. [iCal]
Contributed Talk 2: Joint Belief Tracking and Reward Optimization through Approximate Inference (Talk)
Sat 8:00 a.m. - 8:15 a.m. [iCal]
Contributed Talk 3: Learning Dexterous In-Hand Manipulation (Talk)
Sat 8:15 a.m. - 8:40 a.m. [iCal]
Pieter Abbeel (Talk)
Pieter Abbeel
Sat 8:40 a.m. - 9:00 a.m. [iCal]
Spotlights & Poster Session (Spotlights)
James A Preiss, Alexander Grishin, Ville Kyrki, Pol Moreno Comellas, Akshay Narayan, Tze-Yun Leong, Yongxi Tan, Lilian Weng, Toshiharu Sugawara, Kenneth Young, Tianmin Shu, Jonas Gehring, Ahmad Beirami, Chris Amato, sammie katt, Andrea Baisero, Arseny Kuznetsov, Jan Humplik, Vladimír Petrík
Sat 11:00 a.m. - 11:25 a.m. [iCal]
Peter Stone (Talk)
Peter Stone
Sat 11:25 a.m. - 11:40 a.m. [iCal]
Contributed Talk 4: Differentiable Algorithm Networks: Learning Wrong Models for Wrong Algorithms (Talk)
Sat 11:40 a.m. - 12:05 p.m. [iCal]
Jilles Dibangoye (Talk)
Jilles Dibangoye
Sat 12:35 p.m. - 1:00 p.m. [iCal]
Anca Dragan (Talk)
Anca Dragan
Sat 1:00 p.m. - 2:00 p.m. [iCal]
Panel Discussion
Sat 2:00 p.m. - 3:30 p.m. [iCal]
Poster Session

Author Information

Joni Pajarinen (TU Darmstadt)
Chris Amato (Northeastern University)
Pascal Poupart (University of Waterloo & RBC Borealis AI)
David Hsu (National University of Singapore)

More from the Same Authors