`

Timezone: »

 
Poster
Visual Adversarial Imitation Learning using Variational Models
Rafael Rafailov · Tianhe Yu · Aravind Rajeswaran · Chelsea Finn

Fri Dec 10 08:30 AM -- 10:00 AM (PST) @ None #None

Reward function specification, which requires considerable human effort and iteration, remains a major impediment for learning behaviors through deep reinforcement learning. In contrast, providing visual demonstrations of desired behaviors presents an easier and more natural way to teach agents. We consider a setting where an agent is provided a fixed dataset of visual demonstrations illustrating how to perform a task, and must learn to solve the task using the provided demonstrations and unsupervised environment interactions. This setting presents a number of challenges including representation learning for visual observations, sample complexity due to high dimensional spaces, and learning instability due to the lack of a fixed reward or learning signal. Towards addressing these challenges, we develop a variational model-based adversarial imitation learning (V-MAIL) algorithm. The model-based approach provides a strong signal for representation learning, enables sample efficiency, and improves the stability of adversarial training by enabling on-policy learning. Through experiments involving several vision-based locomotion and manipulation tasks, we find that V-MAIL learns successful visuomotor policies in a sample-efficient manner, has better stability compared to prior work, and also achieves higher asymptotic performance. We further find that by transferring the learned models, V-MAIL can learn new tasks from visual demonstrations without any additional environment interactions. All results including videos can be found online at https://sites.google.com/view/variational-mail

Author Information

Rafael Rafailov (Stanford University)
Tianhe Yu (Stanford University)
Aravind Rajeswaran (FAIR / Facebook AI Research)
Chelsea Finn (Stanford University)

More from the Same Authors