Timezone: »

 
Poster
Learning Dynamic Belief Graphs to Generalize on Text-Based Games
Ashutosh Adhikari · Xingdi Yuan · Marc-Alexandre Côté · Mikuláš Zelinka · Marc-Antoine Rondeau · Romain Laroche · Pascal Poupart · Jian Tang · Adam Trischler · Will Hamilton

Mon Dec 07 09:00 PM -- 11:00 PM (PST) @ Poster Session 0 #37
in Poster Session 1 »

Playing text-based games requires skills in processing natural language and sequential decision making. Achieving human-level performance on text-based games remains an open challenge, and prior research has largely relied on hand-crafted structured representations and heuristics. In this work, we investigate how an agent can plan and generalize in text-based games using graph-structured representations learned end-to-end from raw text. We propose a novel graph-aided transformer agent (GATA) that infers and updates latent belief graphs during planning to enable effective action selection by capturing the underlying game dynamics. GATA is trained using a combination of reinforcement and self-supervised learning. Our work demonstrates that the learned graph-based representations help agents converge to better policies than their text-only counterparts and facilitate effective generalization across game configurations. Experiments on 500+ unique games from the TextWorld suite show that our best agent outperforms text-based baselines by an average of 24.2%.

Author Information

Ashutosh Adhikari (University of Waterloo)
Eric Yuan (Microsoft Research)
Marc-Alexandre Côté (Microsoft Research)
Mikuláš Zelinka (Charles University, Faculty of Mathematics and Physics)
Marc-Antoine Rondeau (Microsoft Research)
Romain Laroche (Microsoft Research)
Pascal Poupart (University of Waterloo & Vector Institute)
Jian Tang (Mila)
Adam Trischler (Microsoft)
Will Hamilton (McGill)

More from the Same Authors