Timezone: »

 
Habitat 2.0: Training Home Assistants to Rearrange their Habitat
Andrew Szot · Alexander Clegg · Eric Undersander · Erik Wijmans · Yili Zhao · Noah Maestre · Mustafa Mukadam · Oleksandr Maksymets · Aaron Gokaslan · Sameer Dharur · Franziska Meier · Wojciech Galuba · Angel Chang · Zsolt Kira · Vladlen Koltun · Jitendra Malik · Manolis Savva · Dhruv Batra

Tue Dec 14 09:00 AM -- 10:00 AM (PST) @
in Ecological Theory of Reinforcement Learning: How Does Task Design Influence Agent Learning? »

We introduce Habitat 2.0 (H2.0), a simulation platform for training virtual robots in interactive 3D environments and complex physics-enabled scenarios. We make comprehensive contributions to all levels of the embodied AI stack – data, simulation, and benchmark tasks. Specifically, we present: (i) ReplicaCAD: an artist-authored, annotated, reconfigurable 3D dataset of apartments (matching real spaces) with articulated objects (e.g. cabinets and drawers that can open/close); (ii) H2.0: a high-performance physics-enabled 3D simulator with speeds exceeding 25,000 simulation steps per second (850× real-time) on an 8-GPU node, representing 100× speed-ups over prior work; and, (iii) Home Assistant Benchmark (HAB): a suite of common tasks for assistive robots (tidy the house, prepare groceries, set the table) that test a range of mobile manipulation capabilities. These large-scale engineering contributions allow us to systematically compare deep reinforcement learning (RL) at scale and classical sense-plan-act (SPA) pipelines in long-horizon structured tasks, with an emphasis on generalization to new objects, receptacles, and layouts. We find that (1) flat RL policies struggle on HAB compared to hierarchical ones; (2) a hierarchy with independent skills suffers from ‘hand-off problems’, and (3) SPA pipelines are more brittle than RL policies.

Author Information

Andrew Szot (Georgia Institute of Technology)
Alexander Clegg (Facebook (FAIR Labs))
Eric Undersander (Facebook)
Erik Wijmans (Georgia Institute of Technology)
Yili Zhao (Facebook AI Research)
Noah Maestre (Facebook)
Mustafa Mukadam (Facebook AI Research)
Oleksandr Maksymets (Facebook AI Research)
Aaron Gokaslan (Cornell)
Sameer Dharur (Georgia Tech)
Franziska Meier (Facebook AI Research)
Wojciech Galuba (Meta AI)
Angel Chang (Simon Fraser University)
Zsolt Kira (Georgia Institute of Techology)
Vladlen Koltun (Apple)
Jitendra Malik (University of California at Berkley)
Manolis Savva (Simon Fraser University)
Dhruv Batra (FAIR (Meta) / Georgia Tech)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors