Timezone: »

 
Distributional deep Q-learning with CVaR regression
Mastane Achab · REDA ALAMI · YASSER ABDELAZIZ DAHOU DJILALI · Kirill Fedyanin · Eric Moulines · Maxim Panov

@
in Deep Reinforcement Learning Workshop »
Event URL: https://openreview.net/forum?id=UqiiZ9Xdlrw »

Reinforcement learning (RL) allows an agent interacting sequentially with an environment to maximize its long-term return, in expectation. In distributional RL (DRL), the agent is also interested in the probability distribution of the return, not just its expected value. This so-called distributional perspective of RL has led to new algorithms with improved empirical performance. In this paper, we recall the atomic DRL (ADRL) framework based on atomic distributions projected via the Wasserstein-2 metric. Then, we derive two new deep ADRL algorithms, namely SAD-Q-learning and MAD-Q-learning (both for the control task). Numerical experiments on various environments compare our approach against existing deep (distributional) RL methods.

[ Poster]  [ OpenReview]  [ Topia] 

Author Information

Mastane Achab (Technology Innovation Institute)
REDA ALAMI (Total)
YASSER ABDELAZIZ DAHOU DJILALI (Dublin City University)
Kirill Fedyanin (Skolkovo Institute of Science and Technology)
Eric Moulines (Ecole Polytechnique)
Maxim Panov (Technology Innovation Institute)

More from the Same Authors