Timezone: »

 
Poster
Forethought and Hindsight in Credit Assignment
Veronica Chelu · Doina Precup · Hado van Hasselt

Tue Dec 08 09:00 AM -- 11:00 AM (PST) @ Poster Session 1 #563
in Poster Session 2 »

We address the problem of credit assignment in reinforcement learning and explore fundamental questions regarding the way in which an agent can best use additional computation to propagate new information, by planning with internal models of the world to improve its predictions. Particularly, we work to understand the gains and peculiarities of planning employed as forethought via forward models or as hindsight operating with backward models. We establish the relative merits, limitations and complementary properties of both planning mechanisms in carefully constructed scenarios. Further, we investigate the best use of models in planning, primarily focusing on the selection of states in which predictions should be (re)-evaluated. Lastly, we discuss the issue of model estimation and highlight a spectrum of methods that stretch from environment dynamics predictors to planner-aware models.

Author Information

Veronica Chelu (McGill University)
Doina Precup (McGill University / Mila / DeepMind Montreal)
Hado van Hasselt (DeepMind)

More from the Same Authors