Timezone: »

 
Poster
Promoting Stochasticity for Expressive Policies via a Simple and Efficient Regularization Method
Qi Zhou · Yufei Kuang · Zherui Qiu · Houqiang Li · Jie Wang

Wed Dec 09 09:00 PM -- 11:00 PM (PST) @ Poster Session 4 #1271
in Poster Session 5 »

Many recent reinforcement learning (RL) methods learn stochastic policies with entropy regularization for exploration and robustness. However, in continuous action spaces, integrating entropy regularization with expressive policies is challenging and usually requires complex inference procedures. To tackle this problem, we propose a novel regularization method that is compatible with a broad range of expressive policy architectures. An appealing feature is that, the estimation of our regularization terms is simple and efficient even when the policy distributions are unknown. We show that our approach can effectively promote the exploration in continuous action spaces. Based on our regularization, we propose an off-policy actor-critic algorithm. Experiments demonstrate that the proposed algorithm outperforms state-of-the-art regularized RL methods in continuous control tasks.

Author Information

Qi Zhou (University of Science and Technology of China)
Yufei Kuang (University of Science and Technology of China)
Zherui Qiu (University of Science and Technology of China)
Houqiang Li (University of Science and Technology of China)
Jie Wang (University of Science and Technology of China)

More from the Same Authors