Timezone: »

Competition Track Saturday
Hugo Jair Escalante · Katja Hofmann

Sat Dec 12 08:00 AM -- 05:45 PM (PST) @ None
Event URL: https://neurips.cc/Conferences/2020/CompetitionTrack »

Second session for the competition program at NeurIPS2020.

Machine learning competitions have grown in popularity and impact over the last decade, emerging as an effective means to advance the state of the art by posing well-structured, relevant, and challenging problems to the community at large. Motivated by a reward or merely the satisfaction of seeing their machine learning algorithm reach the top of a leaderboard, practitioners innovate, improve, and tune their approach before evaluating on a held-out dataset or environment. The competition track of NeurIPS has matured in 2020, its fourth year, with a considerable increase in both the number of challenges and the diversity of domains and topics. A total of 16 competitions are featured this year as part of the track, with 8 competitions associated to each of the two days. The list of competitions that ar part of the program are available here:


Sat 8:00 a.m. - 8:05 a.m.
Introducing EfficientQA: Open domain question answering with memory constraints as a testbed for language understanding and knowledge representations (Live oral presentation)
Tom Kwiatkowski
Sat 8:05 a.m. - 8:25 a.m.
  • smallest question answering system with 25% accuracy
  • best performing system smaller than 500 Mb
  • best performing system smaller than 6Gb
  • best performing system overall
Tom Kwiatkowski
Sat 8:25 a.m. - 8:45 a.m.

Five top human teams of trivia experts took on the competition’s baseline systems for the opportunity to take on the computer systems in each of the competition’s divisions. The team of humans will compete against the computer on thirty questions from the test set. We will present highlights from the preliminary competition as well as the final showdown between computer systems and the human teams.

Jordan Boyd-Graber
Sat 9:00 a.m. - 9:06 a.m.
Predicting Generalization in Deep Learning (PGDL): Opening remark (Intro)
Yiding Jiang
Sat 9:06 a.m. - 9:16 a.m.
Keynote speech: Sanjeev Arora (PGDL) (Oral presentation)
Sanjeev Arora, Yiding Jiang
Sat 9:16 a.m. - 9:17 a.m.
Introduction to winning team (PGDL) (Intro)
Yiding Jiang
Sat 9:17 a.m. - 9:27 a.m.
Winning team presentation: On Representations and Generalization (PGDL) (Oral presentation) Video
Yiding Jiang, Parth Natekar, Manik Sharma
Sat 9:27 a.m. - 9:28 a.m.
Introduction to Runner up 1 (PGDL) (Intro)
Yiding Jiang
Sat 9:28 a.m. - 9:33 a.m.
Runner up presentation: Robustness to Augmentations as a Generalization Metric (PGDL) (Oral presentation) Video
Sumukh Aithal K
Sat 9:33 a.m. - 9:34 a.m.
Introduction to Runner up 2 (PGDL) (Intro)
Yiding Jiang
Sat 9:34 a.m. - 9:39 a.m.
Runner up presentation: Ranking generalization via smoothness of latent graphs (PGDL) (Oral presentation) Video
Carlos Lassance
Sat 9:39 a.m. - 9:44 a.m.
Closing remark (PGDL) (Outro)
Yiding Jiang
Sat 10:00 a.m. - 10:15 a.m.
Openning and design of the INTERPRET challenge @ NeurIPS2020 (Oral presentation)
Wei Zhan
Sat 10:15 a.m. - 10:35 a.m.
Winner talks of INTERPRET challenge (Oral presentation)
Hengbo Ma
Sat 10:35 a.m. - 10:45 a.m.
Analysis, research opportunities and closing of INTERPRET challenge (Oral presentation)
Liting Sun
Sat 11:00 a.m. - 11:05 a.m.
NLC2CMD Competition Organizers: Introduction, Problem Description, CLAI (Oral presentation) Video
Mayank Agarwal
Sat 11:05 a.m. - 11:15 a.m.
NLC2CMD Competition Keynote: Tellina (Oral presentation)
Victoria Lin, Mayank Agarwal, Tathagata Chakraborti
Sat 11:15 a.m. - 11:20 a.m.
NLC2CMD Competition Organizers: Metrics, Data, Tracks (Live oral presentation) Video
Mayank Agarwal
Sat 11:20 a.m. - 11:22 a.m.
NLC2CMD Competition Participant Team: AINixClaiSimple (Oral presentation) Video
David Gros
Sat 11:22 a.m. - 11:24 a.m.
NLC2CMD Competition Participant Team: coinse-team (Oral presentation) Video
Juyeon Yoon
Sat 11:24 a.m. - 11:26 a.m.
NLC2CMD Competition Participant Team: AICore (Oral presentation) Video
Kangwook Lee
Sat 11:26 a.m. - 11:28 a.m.
NLC2CMD Competition Participant Team: magnum (Oral presentation) Video
Quchen Fu
Sat 11:28 a.m. - 11:30 a.m.
NLC2CMD Competition Participant Team: Hubris (Oral presentation) Video
Jaron Maene
Sat 11:30 a.m. - 11:32 a.m.
NLC2CMD Competition Participant Team: jb (Oral presentation) Video
Denis Litvinov
Sat 11:32 a.m. - 11:45 a.m.
NLC2CMD Competition Organizers: Results (Live) (Outro)
Kartik Talamadupula
Sat 2:00 p.m. - 2:03 p.m.
Introduction to AIDO (Intro)
Liam Paull
Sat 2:03 p.m. - 2:05 p.m.
Short Scientific Talk (AIDO) (Oral presentation)
Luigi Di Lillo
Sat 2:05 p.m. - 2:13 p.m.
Advanced Perception League (Oral presentation)
Liam Paull
Sat 2:13 p.m. - 2:19 p.m.
Intro to Urban League (includes highlights from semifinals) (Intro)
Liam Paull
Sat 2:19 p.m. - 2:34 p.m.
Live robot competition (LF, LFP, lFVM) (Live competition)
Liam Paull
Sat 2:34 p.m. - 2:40 p.m.
Interviews with winners (Panel discussion)
Liam Paull
Sat 2:40 p.m. - 2:43 p.m.
Conclusions and Wrap up (Outro)
Liam Paull
Sat 3:00 p.m. - 3:05 p.m.
Introduction - Flatland (Intro)
Sharada Mohanty
Sat 3:05 p.m. - 3:15 p.m.
Flatland Competition Design & Results (Oral presentation) Video
Sharada Mohanty
Sat 3:15 p.m. - 3:19 p.m.
Winner Talks : Team An Old Driver (Oral presentation) Video
Sharada Mohanty
Sat 3:19 p.m. - 3:23 p.m.
Winner Talks : Team JBR_HSE (Oral presentation) Video
Sharada Mohanty
Sat 3:23 p.m. - 3:27 p.m.
Winner Talks : Team ai-team-flatland (Oral presentation) Video
Sharada Mohanty
Sat 3:27 p.m. - 3:42 p.m.
"Real world applications of Flatland" : Panel Discussion with SBB, DeutschBahn, SNCF (Oral presentation) Video
Sharada Mohanty
Sat 3:42 p.m. - 3:45 p.m.
Concluding Remarks (Outro)
Sharada Mohanty
Sat 4:00 p.m. - 4:03 p.m.
Introduction - Procgen (Intro)
Sharada Mohanty
Sat 4:03 p.m. - 4:12 p.m.
Introduction to the Procgen Benchmark (Oral presentation) Video
Karl Cobbe
Sat 4:12 p.m. - 4:20 p.m.
NeurIPS 2020 Procgen Challenge Design (Oral presentation) Video
Sharada Mohanty
Sat 4:20 p.m. - 4:25 p.m.
Winner Announcements & Analysis of top submissions (Oral presentation) Video
Sharada Mohanty
Sat 4:25 p.m. - 4:44 p.m.
Sample Efficiency & Generalization in RL : An assortment of tricks (talks by top participants) (Oral presentation) Video
Sharada Mohanty
Sat 4:44 p.m. - 4:45 p.m.
Concluding Remarks (Outro)
Sharada Mohanty
Sat 5:00 p.m. - 5:45 p.m.
Introduction and results of the 2020 MineRL Competition (Live oral presentation)
William Guss, Stephanie Milani, Nicholay Topin

Author Information

Hugo Jair Escalante (INAOE)
Katja Hofmann (Microsoft Research)

Dr. Katja Hofmann is a Principal Researcher at the [Game Intelligence](http://aka.ms/gameintelligence/) group at [Microsoft Research Cambridge, UK](https://www.microsoft.com/en-us/research/lab/microsoft-research-cambridge/). There, she leads a research team that focuses on reinforcement learning with applications in modern video games. She and her team strongly believe that modern video games will drive a transformation of how we interact with AI technology. One of the projects developed by her team is [Project Malmo](https://www.microsoft.com/en-us/research/project/project-malmo/), which uses the popular game Minecraft as an experimentation platform for developing intelligent technology. Katja's long-term goal is to develop AI systems that learn to collaborate with people, to empower their users and help solve complex real-world problems. Before joining Microsoft Research, Katja completed her PhD in Computer Science as part of the [ILPS](https://ilps.science.uva.nl/) group at the [University of Amsterdam](https://www.uva.nl/en). She worked with Maarten de Rijke and Shimon Whiteson on interactive machine learning algorithms for search engines.

More from the Same Authors