Timezone: »

Self-Imitation Learning via Generalized Lower Bound Q-learning
Yunhao Tang

Tue Dec 08 09:00 AM -- 11:00 AM (PST) @ Poster Session 1 #193

Self-imitation learning motivated by lower-bound Q-learning is a novel and effective approach for off-policy learning. In this work, we propose a n-step lower bound which generalizes the original return-based lower-bound Q-learning, and introduce a new family of self-imitation learning algorithms. To provide a formal motivation for the potential performance gains provided by self-imitation learning, we show that n-step lower bound Q-learning achieves a trade-off between fixed point bias and contraction rate, drawing close connections to the popular uncorrected n-step Q-learning. We finally show that n-step lower bound Q-learning is a more robust alternative to return-based self-imitation learning and uncorrected n-step, over a wide range of benchmark tasks.

Author Information

Yunhao Tang (Columbia University)

I am a PhD student at Columbia IEOR. My research interests are reinforcement learning and approximate inference.

More from the Same Authors