Timezone: »

Train Offline, Test Online: A Democratized Robotics Benchmark
Victoria Dean · Gaoyue Zhou · Mohan Kumar Srirama · Sudeep Dasari · Esther Brown · Marion Lepert · Paul Ruvolo · Chelsea Finn · Lerrel Pinto · Abhinav Gupta

Sat Dec 16 11:30 AM -- 02:30 PM (PST) @ Virtual

The Train Offline, Test Online (TOTO) competition provides a shared, remote robot setup paired with an open-source dataset. Participants can train offline agents (e.g. via behavior cloning or offline reinforcement learning) and evaluate them on two common manipulation tasks (pouring and scooping), which require challenging generalization across objects, locations, and lighting conditions. TOTO has an additional track for evaluating vision representations, which are combined with a standard behavior cloning method for evaluation. The competition begins with a simulation phase to qualify for the real-robot phase. We hope that TOTO will recruit newcomers to robotics by giving them a chance to compete and win on real hardware and the resources needed to get started.

Author Information

Victoria Dean (Olin College of Engineering)
Gaoyue Zhou (Carnegie Mellon University)
Mohan Kumar Srirama (Carnegie Mellon University)
Sudeep Dasari (Carnegie Mellon University)
Esther Brown (Harvard University)
Marion Lepert (Stanford)
Paul Ruvolo (Olin College of Engineering)
Chelsea Finn (Stanford)
Lerrel Pinto (New York University)
Abhinav Gupta (Carnegie Mellon University Robotics Institute)

More from the Same Authors