Timezone: »
We present a deep imitation learning framework for robotic bimanual manipulation in a continuous state-action space. A core challenge is to generalize the manipulation skills to objects in different locations. We hypothesize that modeling the relational information in the environment can significantly improve generalization. To achieve this, we propose to (i) decompose the multi-modal dynamics into elemental movement primitives, (ii) parameterize each primitive using a recurrent graph neural network to capture interactions, and (iii) integrate a high-level planner that composes primitives sequentially and a low-level controller to combine primitive dynamics and inverse kinematics control. Our model is a deep, hierarchical, modular architecture. Compared to baselines, our model generalizes better and achieves higher success rates on several simulated bimanual robotic manipulation tasks. We open source the code for simulation, data, and models at: https://github.com/Rose-STL-Lab/HDR-IL.
Author Information
Fan Xie (Northeastern University)
Alexander Chowdhury (Northeastern University)
Clara De Paolis Kaluza (Northeastern University)
Linfeng Zhao (Northeastern University)
Lawson Wong (Northeastern University)
Rose Yu (University of California, San Diego)
More from the Same Authors
-
2020 Workshop: Machine Learning for Engineering Modeling, Simulation and Design »
Alex Beatson · Priya Donti · Amira Abdel-Rahman · Stephan Hoyer · Rose Yu · J. Zico Kolter · Ryan Adams -
2020 Poster: Learning Disentangled Representations of Videos with Missing Data »
Armand Comas · Chi Zhang · Zlatan Feric · Octavia Camps · Rose Yu -
2020 Session: Orals & Spotlights Track 06: Dynamical Sys/Density/Sparsity »
Animesh Garg · Rose Yu -
2017 Poster: Subset Selection and Summarization in Sequential Data »
Ehsan Elhamifar · M. Clara De Paolis Kaluza