Timezone: »

 
Visual Imitation Learning with Patch Rewards
Minghuan Liu · Tairan He · Weinan Zhang · Shuicheng Yan · Zhongwen Xu

@
in Deep Reinforcement Learning Workshop »
Event URL: https://openreview.net/forum?id=G4APgu4d7v »

Visual imitation learning enables reinforcement learning agents to learn to behave from expert visual demonstrations such as videos or image sequences, without explicit, well-defined rewards. Previous reseaches either adopt supervised learning techniques or induce simple and coarse scalar rewards from pixels, neglecting the dense information contained in the image demonstrations.In this work, we propose to measure the expertise of various local regions of image samples, or called patches, and recover multi-dimensional patch rewards accordingly. Patch reward is a more precise rewarding characterization that serves as fine-grained expertise measurement and visual explainability tool.Specifically, we present Adversarial Imitation Learning with Patch Rewards (PatchAIL), which employs a patch-based discriminator to measure the expertise of different local parts from given images and provide patch rewards.The patch-based knowledge is also used to regularize the aggregated reward and stabilize the training.We evaluate our method on the standard pixel-based benchmark DeepMind Control Suite. The experiment results have demonstrated that PatchAIL outperforms baseline methods and provides valuable interpretations for visual demonstrations.

Author Information

Minghuan Liu (Shanghai Jiao Tong University)
Tairan He (Shanghai Jiao Tong University)
Tairan He

I am an undergraduate student at Shanghai Jiao Tong University (SJTU), majoring in Computer Science & Technology. I have been working as a research intern at APEX Lab since 2019, advised by Prof. Weinan Zhang. I am now a visiting student at Intelligent Control Lab in the Robotics Institute at Carnegie Mellon University, advised by Prof. Changliu Liu. Prior to that, I was research intern at Microsoft Research.

Weinan Zhang (Shanghai Jiao Tong University)
Shuicheng Yan (Sea AI Lab)
Zhongwen Xu (Sea AI Lab)

More from the Same Authors