Timezone: »

 
Spotlight
Multiple-Step Greedy Policies in Approximate and Online Reinforcement Learning
Yonathan Efroni · Gal Dalal · Bruno Scherrer · Shie Mannor

Thu Dec 06 01:15 PM -- 01:20 PM (PST) @ Room 220 CD
in Thu Track 1 -- Session 2 »

Multiple-step lookahead policies have demonstrated high empirical competence in Reinforcement Learning, via the use of Monte Carlo Tree Search or Model Predictive Control. In a recent work (Efroni et al., 2018), multiple-step greedy policies and their use in vanilla Policy Iteration algorithms were proposed and analyzed. In this work, we study multiple-step greedy algorithms in more practical setups. We begin by highlighting a counter-intuitive difficulty, arising with soft-policy updates: even in the absence of approximations, and contrary to the 1-step-greedy case, monotonic policy improvement is not guaranteed unless the update stepsize is sufficiently large. Taking particular care about this difficulty, we formulate and analyze online and approximate algorithms that use such a multi-step greedy operator.

Author Information

Yonathan Efroni (Technion)
Gal Dalal (Technion)
Bruno Scherrer (INRIA)
Shie Mannor (Technion)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors