Human intelligence has the remarkable ability to quickly adapt to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions. To facilitate research in this direction, we propose IGLU: Interactive Grounded Language Understanding in a Collaborative Environment.The primary goal of the competition is to approach the problem of how to develop interactive embodied agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment. Understanding the complexity of the challenge, we split it into sub-tasks to make it feasible for participants. This research challenge is naturally related, but not limited, to two fields of study that are highly relevant to the NeurIPS community: Natural Language Understanding and Generation (NLU/G) and Reinforcement Learning (RL). Therefore, the suggested challenge can bring two communities together to approach one of the important challenges in AI. Another important aspect of the challenge is the dedication to perform a human-in-the-loop evaluation as a final evaluation for the agents developed by contestants.
Tue 3:00 a.m. - 3:05 a.m.
|
Opening Remarks
(
Introduction
)
|
🔗 |
Tue 3:05 a.m. - 3:45 a.m.
|
Towards autonomous interaction with the world wide web - Karthik Narasimhan
(
Recorded Talk
)
Existing benchmarks for grounding language in interactive environments either lack real-world linguistic elements, or prove difficult to scale up due to substantial human involvement in the collection of data or feedback signals. The web provides the perfect balance between the two - it is very practical for the deployment of autonomous agents for helping reduce human effort, while being more scalable than physical setups like robotics. In this talk, I will first describe WebShop, a new RL environment based on a simulated e-commerce website containing >1 million real-world products and >12000 crowd-sourced text instructions. WebShop provides several challenges for language grounding including understanding compositional instructions, query (re-)formulation, comprehending and acting on noisy text in webpages, and performing strategic exploration. Then, I will introduce ReAct, a new approach for large language models (LLMs) to perform both reasoning and acting over the web (or any API) and gain external knowledge for a task. |
🔗 |
Tue 3:45 a.m. - 4:45 a.m.
|
IGLU Competition 2022
(
Talk
)
|
🔗 |
Tue 4:45 a.m. - 5:00 a.m.
|
NLP Track - Felipe Bivort Haiek
(
Spotlight
)
|
🔗 |
Tue 5:00 a.m. - 5:15 a.m.
|
NLP Track - Zhengxiang Shi
(
Spotlight
)
|
🔗 |
Tue 5:15 a.m. - 5:30 a.m.
|
RL Track - Seung Eun Rho
(
Spotlight
)
|
🔗 |
Tue 5:30 a.m. - 5:45 a.m.
|
RL Track - Felipe Bivort Haiek
(
Spotlight
)
|
🔗 |
Tue 5:45 a.m. - 6:00 a.m.
|
RL Track - Edwin Zhang
(
Spotlight
)
|
🔗 |
-
|
Fifteen-minute Competition Overview Video
(
Overview
)
SlidesLive Video » |
Maartje Anne ter Hoeve · Mikhail Burtsev · Zoya Volovikova · Ziming Li · Yuxuan Sun · Shrestha Mohanty · Negar Arabzadeh · Mohammad Aliannejadi · Milagro Teruel · Marc-Alexandre Côté · Kavya Srinet · arthur szlam · Artem Zholus · Alexey Skrynnik · Aleksandr Panov · Ahmed Awadallah · Julia Kiseleva
|