Timezone: »

 
Poster
Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning
Yuanpei Chen · Tianhao Wu · Shengjie Wang · Xidong Feng · Jiechuan Jiang · Zongqing Lu · Stephen McAleer · Hao Dong · Song-Chun Zhu · Yaodong Yang

@

Achieving human-level dexterity is an important open problem in robotics. However, tasks of dexterous hand manipulation even at the baby level are challenging to solve through reinforcement learning (RL). The difficulty lies in the high degrees of freedom and the required cooperation among heterogeneous agents (e.g., joints of fingers). In this study, we propose the Bimanual Dexterous Hands Benchmark (Bi-DexHands), a simulator that involves two dexterous hands with tens of bimanual manipulation tasks and thousands of target objects. Tasks in Bi-DexHands are first designed to match human-level motor skills according to literature in cognitive science, and then are built in Issac Gym; this enables highly efficient RL trainings, reaching 30,000+ FPS by only one single NVIDIA RTX 3090. We provide a comprehensive benchmark for popular RL algorithms under different settings; this includes multi-agent RL, offline RL, multi-task RL, and meta RL. Our results show that PPO type on-policy algorithms can learn to solve simple manipulation tasks that are equivalent up to 48-month human baby (e.g., catching a flying object, opening a bottle), while multi-agent RL can further help to learn manipulations that require skilled bimanual cooperation (e.g., lifting a pot, stacking blocks). Despite the success on each individual task, when it comes to mastering multiple manipulation skills, existing RL algorithms fail to work in most of the multi-task and the few-shot learning tasks, which calls for more future development from the RL community. Our project is open-sourced at https://github.com/PKU-MARL/DexterousHands.

Keywords: Multi-Agent RL · Reinforcement Learning · Robotics
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Yuanpei Chen (South China University of Technology)
Yuanpei Chen

I am a four-year undergraduate student at South China University of Technology, working with Prof. Yaodong Yang at Peking University. I also had the privilege of working closely with Prof. Chenguang Yang at SCUT and Prof. Hao Dong at Peking University.

Tianhao Wu (Peking University)
Shengjie Wang (Tsinghua University, Tsinghua University)
Xidong Feng (University College London)
Jiechuan Jiang (Peking University)
Zongqing Lu (Peking University)
Stephen McAleer (CMU)
Hao Dong (Peking University)
Song-Chun Zhu (UCLA)
Yaodong Yang (AIG)

More from the Same Authors