ilpyt: Imitation Learning Research Code Base in PyTorch

Imitation learning, or learning by example, is an intuitive way to teach new behaviors to autonomous systems. With the parallel growth of deep reinforcement learning research, a rich taxonomy of imitation learning algorithms has emerged. These imitation learning algorithms show promise for teaching safe robot behaviors in increasingly dynamic environments by (1) implicitly bounding behaviors to lay in the field of human demonstration and (2) tackling the computational scalability issues of modern reinforcement learning methods. In this paper, we present ilpyt, a research code base which implements a variety of imitation learning and reinforcement learning algorithm families in a shared infrastructure. It contains implementations of popular deep imitation learning algorithms, written in a modular fashion for easy user customization, novel implementation, and fast benchmarking. The provided algorithm implementations were done in Python using PyTorch, and the overall library organization is inspired by the popular reinforcement learning research library, rlpyt. This white paper summarizes the key features and basic usage of the ilpyt library, as well as benchmark results for the implemented algorithms in several representative OpenAI Gym environments. We hope ilpyt can serve as a launching point for accelerated development in the imitation learning field. ilpyt is available for download at https://github.com/mitre/ilpyt.