Timezone: »

 
Poster
Exploration via Hindsight Goal Generation
Zhizhou Ren · Kefan Dong · Yuan Zhou · Qiang Liu · Jian Peng

Tue Dec 10 05:30 PM -- 07:30 PM (PST) @ East Exhibition Hall B + C #203
in Reinforcement Learning and Planning -- Reinforcement Learning »

Goal-oriented reinforcement learning has recently been a practical framework for robotic manipulation tasks, in which an agent is required to reach a certain goal defined by a function on the state space. However, the sparsity of such reward definition makes traditional reinforcement learning algorithms very inefficient. Hindsight Experience Replay (HER), a recent advance, has greatly improved sample efficiency and practical applicability for such problems. It exploits previous replays by constructing imaginary goals in a simple heuristic way, acting like an implicit curriculum to alleviate the challenge of sparse reward signal. In this paper, we introduce Hindsight Goal Generation (HGG), a novel algorithmic framework that generates valuable hindsight goals which are easy for an agent to achieve in the short term and are also potential for guiding the agent to reach the actual goal in the long term. We have extensively evaluated our goal generation algorithm on a number of robotic manipulation tasks and demonstrated substantially improvement over the original HER in terms of sample efficiency.

Keywords: Reinforcement Learning · Exploration · Reinforcement Learning and Planning
[ Paper]  [ Slides] 

Author Information

Zhizhou Ren (Tsinghua University)
Kefan Dong (Tsinghua University)
Yuan Zhou (UIUC)
Qiang Liu (UT Austin)
Jian Peng (University of Illinois at Urbana-Champaign)

More from the Same Authors