Timezone: »
Human intelligence has the remarkable ability to quickly adapt to new tasks and environments. Starting from a very young age, humans acquire new skills and learn how to solve new tasks either by imitating the behavior of others or by following provided natural language instructions. To facilitate research in this direction, we propose IGLU: Interactive Grounded Language Understanding in a Collaborative Environment.The primary goal of the competition is to approach the problem of how to develop interactive embodied agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment. Understanding the complexity of the challenge, we split it into sub-tasks to make it feasible for participants. This research challenge is naturally related, but not limited, to two fields of study that are highly relevant to the NeurIPS community: Natural Language Understanding and Generation (NLU/G) and Reinforcement Learning (RL). Therefore, the suggested challenge can bring two communities together to approach one of the important challenges in AI. Another important aspect of the challenge is the dedication to perform a human-in-the-loop evaluation as a final evaluation for the agents developed by contestants.
Tue 3:00 a.m. - 3:05 a.m.
|
Opening Remarks
(
Introduction
)
|
🔗 |
Tue 3:05 a.m. - 3:45 a.m.
|
Towards autonomous interaction with the world wide web - Karthik Narasimhan
(
Recorded Talk
)
Existing benchmarks for grounding language in interactive environments either lack real-world linguistic elements, or prove difficult to scale up due to substantial human involvement in the collection of data or feedback signals. The web provides the perfect balance between the two - it is very practical for the deployment of autonomous agents for helping reduce human effort, while being more scalable than physical setups like robotics. In this talk, I will first describe WebShop, a new RL environment based on a simulated e-commerce website containing >1 million real-world products and >12000 crowd-sourced text instructions. WebShop provides several challenges for language grounding including understanding compositional instructions, query (re-)formulation, comprehending and acting on noisy text in webpages, and performing strategic exploration. Then, I will introduce ReAct, a new approach for large language models (LLMs) to perform both reasoning and acting over the web (or any API) and gain external knowledge for a task. |
🔗 |
Tue 3:45 a.m. - 4:45 a.m.
|
IGLU Competition 2022
(
Talk
)
|
🔗 |
Tue 4:45 a.m. - 5:00 a.m.
|
NLP Track - Felipe Bivort Haiek
(
Spotlight
)
|
🔗 |
Tue 5:00 a.m. - 5:15 a.m.
|
NLP Track - Zhengxiang Shi
(
Spotlight
)
|
🔗 |
Tue 5:15 a.m. - 5:30 a.m.
|
RL Track - Seung Eun Rho
(
Spotlight
)
|
🔗 |
Tue 5:30 a.m. - 5:45 a.m.
|
RL Track - Felipe Bivort Haiek
(
Spotlight
)
|
🔗 |
Tue 5:45 a.m. - 6:00 a.m.
|
RL Track - Edwin Zhang
(
Spotlight
)
|
🔗 |
-
|
Fifteen-minute Competition Overview Video
(
Overview
)
SlidesLive Video » |
Maartje Anne ter Hoeve · Mikhail Burtsev · Zoya Volovikova · Ziming Li · Yuxuan Sun · Shrestha Mohanty · Negar Arabzadeh · Mohammad Aliannejadi · Milagro Teruel · Marc-Alexandre Côté · Kavya Srinet · arthur szlam · Artem Zholus · Alexey Skrynnik · Aleksandr Panov · Ahmed Awadallah · Julia Kiseleva
|
Author Information
Julia Kiseleva (Microsoft Research)
Alexey Skrynnik (Artificial Intelligence Research Institute)
Artem Zholus (Moscow Institute of Physics and Technology)
Shrestha Mohanty (Microsoft)
Negar Arabzadeh (University of Waterloo)
Marc-Alexandre Côté (Microsoft Research)
Mohammad Aliannejadi (University of Amsterdam)
Milagro Teruel (FAMAF - Universidad Nacional de Córdoba)
Ziming Li (University of Amsterdam)
Mikhail Burtsev (Artificial Intelligence Research Institute (AIRI))
Maartje Anne ter Hoeve (University of Amsterdam)
Zoya Volovikova (Individual Contributor)
Aleksandr Panov (Artificial Intelligence Institute AIRI)
Yuxuan Sun (Meta AI)
arthur szlam (Facebook)
Ahmed Awadallah (Microsoft)
Kavya Srinet (Facebook AI Research)
More from the Same Authors
-
2021 Spotlight: Hash Layers For Large Sparse Models »
Stephen Roller · Sainbayar Sukhbaatar · arthur szlam · Jason Weston -
2022 : Learning to Reason and Memorize with Self-Questioning »
Jack Lanchantin · Shubham Toshniwal · Jason E Weston · arthur szlam · Sainbayar Sukhbaatar -
2022 : Fifteen-minute Competition Overview Video »
Maartje Anne ter Hoeve · Mikhail Burtsev · Zoya Volovikova · Ziming Li · Yuxuan Sun · Shrestha Mohanty · Negar Arabzadeh · Mohammad Aliannejadi · Milagro Teruel · Marc-Alexandre Côté · Kavya Srinet · arthur szlam · Artem Zholus · Alexey Skrynnik · Aleksandr Panov · Ahmed Awadallah · Julia Kiseleva -
2022 : Learning to Reason and Memorize with Self-Questioning »
Jack Lanchantin · Shubham Toshniwal · Jason E Weston · arthur szlam · Sainbayar Sukhbaatar -
2022 Workshop: LaReL: Language and Reinforcement Learning »
Laetitia Teodorescu · Laura Ruis · Tristan Karch · Cédric Colas · Paul Barde · Jelena Luketina · Athul Jacob · Pratyusha Sharma · Edward Grefenstette · Jacob Andreas · Marc-Alexandre Côté -
2022 Poster: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models »
Dongkuan (DK) Xu · Subhabrata Mukherjee · Xiaodong Liu · Debadeepta Dey · Wenhui Wang · Xiang Zhang · Ahmed Awadallah · Jianfeng Gao -
2021 Poster: Hash Layers For Large Sparse Models »
Stephen Roller · Sainbayar Sukhbaatar · arthur szlam · Jason Weston -
2021 : IGLU: Interactive Grounded Language Understanding in a Collaborative Environment + Q&A »
Julia Kiseleva · Ziming Li · Mohammad Aliannejadi · Maartje Anne ter Hoeve · Mikhail Burtsev · Alexey Skrynnik · Artem Zholus · Aleksandr Panov · Katja Hofmann · Kavya Srinet · arthur szlam · Michel Galley · Ahmed Awadallah -
2020 Workshop: Wordplay: When Language Meets Games »
Prithviraj Ammanabrolu · Matthew Hausknecht · Xingdi Yuan · Marc-Alexandre Côté · Adam Trischler · Kory Mathewson @korymath · John Urbanek · Jason Weston · Mark Riedl -
2020 Poster: Graph Policy Network for Transferable Active Learning on Graphs »
Shengding Hu · Zheng Xiong · Meng Qu · Xingdi Yuan · Marc-Alexandre Côté · Zhiyuan Liu · Jian Tang -
2020 Poster: Uncertainty-aware Self-training for Few-shot Text Classification »
Subhabrata Mukherjee · Ahmed Awadallah -
2020 Spotlight: Uncertainty-aware Self-training for Few-shot Text Classification »
Subhabrata Mukherjee · Ahmed Awadallah -
2020 Poster: Learning Dynamic Belief Graphs to Generalize on Text-Based Games »
Ashutosh Adhikari · Xingdi Yuan · Marc-Alexandre Côté · Mikuláš Zelinka · Marc-Antoine Rondeau · Romain Laroche · Pascal Poupart · Jian Tang · Adam Trischler · Will Hamilton -
2019 : The MineRL competition »
Misa Ogura · Joe Booth · Sophia Sun · Nicholay Topin · Brandon Houghton · William Guss · Stephanie Milani · Oriol Vinyals · Katja Hofmann · JIA KIM · Karolis Ramanauskas · Florian Laurent · Daichi Nishio · Anssi Kanervisto · Alexey Skrynnik · Artemij Amiranashvili · Christian Scheller · KAIXIN WANG · Yanick Schraner -
2019 : Poster Session #2 »
Yunzhu Li · Peter Meltzer · Jianing Sun · Guillaume SALHA · Marin Vlastelica Pogančić · Chia-Cheng Liu · Fabrizio Frasca · Marc-Alexandre Côté · Vikas Verma · Abdulkadir CELIKKANAT · Pierluca D'Oro · Priyesh Vijayan · Maria Schuld · Petar Veličković · Kshitij Tayal · Yulong Pei · Hao Xu · Lei Chen · Pengyu Cheng · Ines Chami · Dongkwan Kim · Guilherme Gomes · Lukasz Maziarka · Jessica Hoffmann · Ron Levie · Antonia Gogoglou · Shunwang Gong · Federico Monti · Wenlin Wang · Yan Leng · Salvatore Vivona · Daniel Flam-Shepherd · Chester Holtz · Li Zhang · MAHMOUD KHADEMI · I-Chung Hsieh · Aleksandar Stanić · Ziqiao Meng · Yuhang Jiao -
2019 Poster: Unsupervised State Representation Learning in Atari »
Ankesh Anand · Evan Racah · Sherjil Ozair · Yoshua Bengio · Marc-Alexandre Côté · R Devon Hjelm -
2018 : Introducing "First TextWorld Problems": a text-based game competition »
Marc-Alexandre Côté -
2018 Workshop: Wordplay: Reinforcement and Language Learning in Text-based Games »
Adam Trischler · Angeliki Lazaridou · Yonatan Bisk · Wendy Tay · Nate Kushman · Marc-Alexandre Côté · Alessandro Sordoni · Daniel Ricks · Tom Zahavy · Hal Daumé III -
2018 Demonstration: TextWorld: A Learning Environment for Text-based Games »
Marc-Alexandre Côté · Wendy Tay · Xingdi Yuan -
2017 Poster: Z-Forcing: Training Stochastic Recurrent Networks »
Anirudh Goyal · Alessandro Sordoni · Marc-Alexandre Côté · Nan Rosemary Ke · Yoshua Bengio -
2017 Tutorial: Geometric Deep Learning on Graphs and Manifolds »
Michael Bronstein · Joan Bruna · arthur szlam · Xavier Bresson · Yann LeCun -
2016 Poster: The Product Cut »
Thomas Laurent · James von Brecht · Xavier Bresson · arthur szlam -
2016 Poster: Learning Multiagent Communication with Backpropagation »
Sainbayar Sukhbaatar · arthur szlam · Rob Fergus -
2015 Poster: End-To-End Memory Networks »
Sainbayar Sukhbaatar · arthur szlam · Jason Weston · Rob Fergus -
2015 Oral: End-To-End Memory Networks »
Sainbayar Sukhbaatar · arthur szlam · Jason Weston · Rob Fergus -
2015 Poster: Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks »
Emily Denton · Soumith Chintala · arthur szlam · Rob Fergus