A 3D Simulator for Evaluating Reinforcement and Imitation Learning Algorithms on Complex Tasks

We present the MASH simulator, a benchmarking tool for both reinforcement and imitation learning algorithms. It comprises a 3D simulator which allows for evaluation on tasks based on motions in a physical environment whose complexities go well beyond that of typical 2D mazes. The simulator is composed of a graphics engine, Ogre 3D, to generate images corresponding to an avatar's views, and of a physical engine, Bullet, to simulate the avatar's physical interactions with the environment when moving around. We have developed a series of tasks and environments within this framework. To each is associated a reward function, and for some of them a teacher can be queried for the optimal action. The framework has been designed so that new tasks and environments can be easily created. This simulator has been developed in C++ and the code is publicly available under the GPLv3 open-source license.