Deploying Imitation Learning using VR Hand Tracking in Robot Manipulation Tasks

Imitation learning is emerging as one of the promising approaches for enabling robots to acquire abilities. Since imitation learning provides methods of learning policies through imitation of an expert’s behavior, it requires sophisticated and sufficient expert behavior trajectories. However, current interfaces for imitation such as kinesthetic teaching or remote operation considerably restrict the ability to efficiently collect diverse demonstration data. To address this challenge, this work proposes an alternative interface for imitation, which can easily transfer human motions to robots while simplifying the demo collection process using a VR-based hand tracking solution. In addition, a novel method that performs data augmentation on expert trajectories is proposed to improve imitation performance. Experimental results showed that the proposed method is effective in collecting expert demonstrations and augmenting the expert trajectories and successfully completing robot manipulation tasks.