Skip to yearly menu bar Skip to main content

Workshop: 4th Robot Learning Workshop: Self-Supervised and Lifelong Learning

IL-flOw: Imitation Learning from Observation using Normalizing Flows

Wei-Di Chang · Juan Camilo Gamboa Higuera · Scott Fujimoto · David Meger · Gregory Dudek


We present an algorithm for Inverse Reinforcement Learning (IRL) from expert state observations only that decouples reward modelling from policy learning, unlike state-of-the-art adversarial methods which require updating the reward model during policy search and are known to be unstable and difficult to optimize. Our method, IL-flOw, recovers the expert policy by modelling state-state transitions, by generating rewards using deep density estimators trained on the demonstration trajectories, avoiding the instability issues of adversarial methods. We demonstrate that using the state transition log-probability density as a reward signal for forward reinforcement learning translates to matching the trajectory distribution of the expert demonstrations, and experimentally show good recovery of the true reward signal as well as state of the art results for imitation from observation on locomotion and robotic continuous control tasks.

Chat is not available.