Timezone: »

 
Simple Emergent Action Representations from Multi-Task Policy Training
Pu Hua · Yubei Chen · Huazhe Xu

@
in Deep Reinforcement Learning Workshop »
Event URL: https://openreview.net/forum?id=Itusjd4iPw »

Low-level sensory and motor signals in the high-dimensional spaces (e.g., image observations or motor torques) in deep reinforcement learning are complicated to understand or harness for downstream tasks directly. While sensory representations have been widely studied, the representations of actions that form motor skills are yet under exploration. In this work, we find that when a multi-task policy network takes as input states and task embeddings, a space based on the task embeddings emerges to contain meaningful action representations with moderate constraints. Within this space, interpolated or composed embeddings can serve as a high-level interface to instruct the agent to perform meaningful action sequences. Empirical results not only show that the proposed action representations have efficacy for intra-action interpolation and inter-action composition with limited or no learning, but also demonstrate their superior ability in task adaptation to strong baselines in Mujoco locomotion tasks. The evidence elucidates that learning action representations is a promising direction toward efficient, adaptable, and composable RL, forming the basis of abstract action planning and the understanding of motor signal space. Anonymous project page: https://sites.google.com/view/emergent-action-representation

[ Poster]  [ OpenReview]  [ Topia] 

Author Information

Pu Hua (Electronic Engineering, Tsinghua University, Tsinghua University)
Yubei Chen (Data Science, NYU Meta AI (FAIR))
Yubei Chen

I received my bachelor’s degree from the Electrical Engineering department at Tsinghua University, Beijing, in 2012. Then, I joined the EECS department and Berkeley AI Research (BAIR) at UC Berkeley to pursue my Ph.D. study on unsupervised learning and generative models, advised by Professor Bruno Olshausen. Along the way, I received my M.S. degree in EECS and M.A. degree in mathematics at Berkeley. In 2012, I was awarded the NSF GRFP fellowship. In 2019, I got my Ph.D. from the EECS department. In late 2020, I started to work with Yann LeCun at Meta AI (FAIR) and the Center for Data Science at NYU as a postdoctoral scholar, where I continued to work on unsupervised representation learning. I also serve as a reviewer for NeurIPS, ICLR, ICML, AAAI, CVPR, ECCV, Neural Computation, etc.

Huazhe Xu (Tsinghua University)

More from the Same Authors