Timezone: »

 
Poster
Uni[MASK]: Unified Inference in Sequential Decision Problems
Micah Carroll · Orr Paradise · Jessy Lin · Raluca Georgescu · Mingfei Sun · David Bignell · Stephanie Milani · Katja Hofmann · Matthew Hausknecht · Anca Dragan · Sam Devlin

Wed Nov 30 09:00 AM -- 11:00 AM (PST) @ Hall J #107
in Poster Session 3 »

Randomly masking and predicting word tokens has been a successful approach in pre-training language models for a variety of downstream tasks. In this work, we observe that the same idea also applies naturally to sequential decision making, where many well-studied tasks like behavior cloning, offline RL, inverse dynamics, and waypoint conditioning correspond to different sequence maskings over a sequence of states, actions, and returns. We introduce the UniMASK framework, which provides a unified way to specify models which can be trained on many different sequential decision making tasks. We show that a single UniMASK model is often capable of carrying out many tasks with performance similar to or better than single-task models. Additionally, after fine-tuning, our UniMASK models consistently outperform comparable single-task models.

Keywords: Deep Learning · Unsupervised Learning · Reinforcement Learning · Multi-task Learning
[ Paper]  [ Poster]  [ OpenReview] 

Author Information

Micah Carroll (UC Berkeley)
Orr Paradise (University of California, Berkeley)
Jessy Lin (University of California Berkeley)
Raluca Georgescu (Microsoft)
Mingfei Sun (Microsoft Research)
David Bignell (Research, Microsoft)
Stephanie Milani (Carnegie Mellon University)
Katja Hofmann (Microsoft Research)

Dr. Katja Hofmann is a Principal Researcher at the [Game Intelligence](http://aka.ms/gameintelligence/) group at [Microsoft Research Cambridge, UK](https://www.microsoft.com/en-us/research/lab/microsoft-research-cambridge/). There, she leads a research team that focuses on reinforcement learning with applications in modern video games. She and her team strongly believe that modern video games will drive a transformation of how we interact with AI technology. One of the projects developed by her team is [Project Malmo](https://www.microsoft.com/en-us/research/project/project-malmo/), which uses the popular game Minecraft as an experimentation platform for developing intelligent technology. Katja's long-term goal is to develop AI systems that learn to collaborate with people, to empower their users and help solve complex real-world problems. Before joining Microsoft Research, Katja completed her PhD in Computer Science as part of the [ILPS](https://ilps.science.uva.nl/) group at the [University of Amsterdam](https://www.uva.nl/en). She worked with Maarten de Rijke and Shimon Whiteson on interactive machine learning algorithms for search engines.

Matthew Hausknecht (Microsoft Research)
Anca Dragan (UC Berkeley)
Sam Devlin (Microsoft Research)

More from the Same Authors