NeurIPS Competitions – Evolution and Opportunities

Abstract: Competitions are set to play a bigger role in research to advance the state-of-the-art in algorithms and help solve scientific and societal challenges. Over the last 15 years, challenges in Machine Learning, Data Science and Artificial Intelligence have proven useful in education to train students and in industry to upskill workers or bring solutions otherwise confined to research. However, with the fast-paced advances in AI, what is the future role of research competitions?

In the talk, we will reflect upon the past 6 years of NeurIPS competitions, highlighting how research competitions are a powerful means of prototyping solutions to novel problems. We will propose an evolution towards engaging a broader diversity of organizers and participants and addressing real world problems experienced by global and local communities. Such evolution may lead the community to rethink how we monitor progress on those fronts and how we can better leverage the wealth of competition outcomes.

Isabelle Guyon recently joined Google Brain as a research scientist. She is also professor of artificial intelligence at Université Paris-Saclay (Orsay). Her areas of expertise include computer vision, bioinformatics, and power systems. She is best known for being a co-inventor of Support Vector Machines. Her recent interests are in automated machine learning, meta-learning, and data-centric AI. She has been a strong promoter of challenges and benchmarks, and is president of ChaLearn, a non-profit dedicated to organizing machine learning challenges. She is community lead of Codalab competitions, a challenge platform used both in academia and industry, which started as a Microsoft research project, under Evelyne Viegas. She co-organized the “Challenges in Machine Learning Workshop” @ NeurIPS between 2014 and 2019 with Evelyne Viegas and others, launched the "NeurIPS challenge track" in 2017 while she was general chair, and pushed the creation of the "NeurIPS datasets and benchmark track" in 2021, as a NeurIPS board member.

Evelyne Viegas is Senior Director of Global Research Engagement at Microsoft Research. She drives the research engagement strategy, designing a portfolio of programs which are open and collaborative with the external research community. She leads initiatives to create social impact through strategic partnerships which accelerate research and technology breakthroughs, impact from research to business and diversification of research partners, ideas and portfolio in the areas of Artificial Intelligence, Computing Systems and Experiences, working in partnership with the academic community, engineering and business groups, industry partners and government agencies worldwide. She co-funded the Challenges in Machine Learning (CiML) Workshop with Isabelle Guyon and co-organized the CiML series at NeurIPS between 2014 and 2019 with her and others to bring the community of competition organizers and participants together to share their efforts and work together on developing the science behind the competitions.