CiML 2019: Machine Learning Competitions for All
Adrienne Mendrik · Wei-Wei Tu · Wei-Wei Tu · Isabelle Guyon · Evelyne Viegas · Ming LI

Fri Dec 13th 08:00 AM -- 06:00 PM @ West 215 + 216
Event URL: »

Challenges in machine learning and data science are open online competitions that address problems by providing datasets or simulated environments. They measure the performance of machine learning algorithms with respect to a given problem. The playful nature of challenges naturally attracts students, making challenges a great teaching resource. However, in addition to the use of challenges as educational tools, challenges have a role to play towards a better democratization of AI and machine learning. They function as cost effective problem-solving tools and a means of encouraging the development of re-usable problem templates and open-sourced solutions. However, at present, the geographic, sociological repartition of challenge participants and organizers is very biased. While recent successes in machine learning have raised much hopes, there is a growing concern that the societal and economical benefits might increasingly be in the power and under control of a few.

CiML (Challenges in Machine Learning) is a forum that brings together workshop organizers, platform providers, and participants to discuss best practices in challenge organization and new methods and application opportunities to design high impact challenges. Following the success of previous years' workshops, we will reconvene and discuss new opportunities for broadening our community.

For this sixth edition of the CiML workshop at NeurIPS our objective is twofold: (1) We aim to enlarge the community, fostering diversity in the community of participants and organizers; (2) We aim to promote the organization of challenges for the benefit of more diverse communities.

The workshop provides room for discussion on these topics and aims to bring together potential partners to organize such challenges and stimulate "machine learning for good", i.e. the organization of challenges for the benefit of society. We have invited prominent speakers that have experience in this domain.

08:00 AM Welcome and Opening Remarks (Opening) Adrienne Mendrik, Wei-Wei Tu, Isabelle Guyon, Evelyne Viegas, Ming LI
08:15 AM Amir Banifatemi (XPrize) "AI for Good via Machine Learning Challenges" (Invited Talk) Amir Banifatemi
09:00 AM Emily Bender (University of Washington) "Making Stakeholder Impacts Visible in the Evaluation Cycle: Towards Fairness-Integrated Shared Tasks and Evaluation Metrics" (Invited Talk) Emily M. Bender
09:45 AM Coffee Break (Break)
10:30 AM Dina Machuve (Nelson Mandela African Institution of Science and Technology) “Machine Learning Competitions: The Outlook from Africa” (Invited Talk) Dina Machuve
11:15 AM Dog Image Generation Competition on Kaggle (Talk) Wendy Kan, Phil Culliton
11:30 AM Learning To Run a Power Network Competition (Talk) Benjamin Donnot
11:45 AM The AI Driving Olympics: An Accessible Robot Learning Benchmark (Talk) Matthew Walter
12:00 PM Conclusion on TrackML, a Particle Physics Tracking Machine Learning Challenge Combining Accuracy and Inference Speed (Talk) David Rousseau, jean-roch vlimant
12:15 PM Catered Lunch and Poster Viewing (in Workshop Room) (Break, Poster Session)
Gustavo Stolovitzky, Prabhu Pradhan, Pablo Duboue, Zhiwen Tang, Aleksei Natekin, Elizabeth Bondi, Xavier Bouthillier, Stephanie Milani, Heimo Müller, Andreas T. Holzinger, Stefan Harrer, Ben Day, Andrey Ustyuzhanin, William Guss, Mahtab Mirmomeni
02:00 PM Frank Hutter (University of Freiburg) "A Proposal for a New Competition Design Emphasizing Scientific Insights" (Invited Talk) Frank Hutter
02:45 PM Design and Analysis of Experiments: A Challenge Approach in Teaching (Talk) Adrien Pavao
03:00 PM The model-to-data paradigm: overcoming data access barriers in biomedical competitions (Talk) Justin Guinney
03:15 PM The Deep Learning Epilepsy Detection Challenge: Design, Implementation, and Test of a New Crowd-Sourced AI Challenge Ecosystem (Talk) Isabell Kiral
03:30 PM Coffee Break (Break)
04:15 PM Open Space Topic “The Organization of Challenges for the Benefit of More Diverse Communities” (Open Space Session) Adrienne Mendrik, Isabelle Guyon, Wei-Wei Tu, Evelyne Viegas, Ming LI

Author Information

Adrienne Mendrik (Netherlands eScience Center)
Wei-Wei Tu (4Paradigm Inc.)
Wei-Wei Tu (4Paradigm Inc.)
Isabelle Guyon (UPSud, INRIA, University Paris-saclay and ChaLearn)
Evelyne Viegas (Microsoft Research)
Ming LI (Nanjing University)

More from the Same Authors