Timezone: »

Machine Learning Challenges as a Research Tool
Isabelle Guyon · Evelyne Viegas · Sergio Escalera · Jacob D Abernethy

Sat Dec 09 08:00 AM -- 06:30 PM (PST) @ S1
Event URL: http://ciml.chalearn.org/ciml2017 »

Challenges in machine learning and data science are competitions running over several weeks or months to resolve problems using provided datasets or simulated environments. The playful nature of challenges naturally attracts students, making challenge a great teaching resource. For this fourth edition of the CiML workshop at NIPS we want to explore the impact of machine learning challenges as a research tool. The workshop will give a large part to discussions around several axes: (1) benefits and limitations of challenges as a research tool; (2) methods to induce and train young researchers; (3) experimental design to foster contributions that will push the state of the art.
CiML is a forum that brings together workshop organizers, platform providers, and participants to discuss best practices in challenge organization and new methods and application opportunities to design high impact challenges. Following the success of last year's workshop, in which a fruitful exchange led to many innovations, we propose to reconvene and discuss new opportunities for challenges as a research tool, one of the hottest topics identified in last year's discussions. We have invited prominent speakers in this field.
We will also reserve time to an open discussion to dig into other topic including open innovation, collaborative challenges (coopetitions), platform interoperability, and tool mutualisation.
The audience of this workshop is targeted to workshop organizers, participants, and anyone with scientific problem involving machine learning, which may be formulated as a challenge. The emphasis of the workshop is on challenge design. Hence it complements nicely the workshop on the NIPS 2017 competition track and will help paving the way toward next year's competition program.

Sat 8:00 a.m. - 8:10 a.m.
Introduction - Isabelle Guyon and Evelyne Viegas (Announcement)
Isabelle Guyon
Sat 8:10 a.m. - 8:40 a.m.
Baázs Kégl, RAMP platform (Invited talk)
Balázs Kégl
Sat 8:40 a.m. - 9:10 a.m.
Automatic evaluation of chatbots - Varvara Logacheva (Talk)
Sat 9:10 a.m. - 9:40 a.m.
TrackML - David Rousseau (Talk)
Sat 9:40 a.m. - 10:10 a.m.
Data Science Bowl - Andre Farris (Talk)
Drew Farris
Sat 10:10 a.m. - 10:40 a.m.
CrowdAI - Mohanty Sarada (Invited talk)
Sat 10:40 a.m. - 11:00 a.m.
Break, poster viewing 1 (Poster)
Sat 11:00 a.m. - 11:30 a.m.
Ben Hamner, Kaggle platform (Invited talk)
Ben Hamner
Sat 11:30 a.m. - 12:20 p.m.
Incentivizing productivity in data science - Jacob Abernethy (Discussion)
Sat 12:20 p.m. - 1:30 p.m.
Establishing Uniform Image Segmentation - Maximilian Chen and Michael C. Darling (Poster)
Max Chen, Michael Darling
Sat 12:20 p.m. - 1:30 p.m.
Lunch Break, poster viewing 2 (Poster)
Sat 1:30 p.m. - 1:40 p.m.
Welcome - Isabelle Guyon and Evelyne Viegas (Announcement)
Sat 1:40 p.m. - 2:10 p.m.
Project Malmo, Minecraft - Katja Hofmann (Invited talk)
Sat 2:10 p.m. - 2:40 p.m.
Project Alloy - Laura Seaman and Israel Ridgley (Talk)
Israel Ridgley, Laura Seaman
Sat 2:40 p.m. - 3:10 p.m.
Education and public service - Jonathan C. Stroud (Talk)
Sat 3:10 p.m. - 3:30 p.m.
Break, poster viewing 3 (Poster)
Sat 3:30 p.m. - 4:00 p.m.
AutoDL (Google challenge) - Olivier Bousquet (Talk)
Sat 4:00 p.m. - 4:30 p.m.
Scoring rule markets - Rafael Frongillo and Bo Waggoner (Talk)
Sat 4:30 p.m. - 5:00 p.m.
ENCODE-DREAM challenge - Akshay Balsubramani (Talk)
Sat 5:00 p.m. - 5:30 p.m.
Codalab platform, Xavier Baro (Invited talk)
Sat 5:30 p.m. - 6:20 p.m.
New opportunities for challenges, looking to the future - Sergio Escalera (Discussion)
Sat 6:20 p.m. - 6:30 p.m.
Wrap up -Evelyne Viegas and Isabelle Guyon (Announcement)

Author Information

Isabelle Guyon (U. Paris-Saclay & ChaLearn)
Evelyne Viegas (Microsoft Research)
Sergio Escalera (University of Barcelona and Computer Vision Center)

Sergio Escalera obtained the P.h.D. degree on Multi-class visual categorization systems at Computer Vision Center, UAB. He obtained the 2008 best Thesis award on Computer Science at Universitat Autònoma de Barcelona. He leads the Human Pose Recovery and Behavior Analysis Group at UB, CVC, and the Barcelona Graduate School of Mathematics. He is an associate professor at the Department of Mathematics and Informatics, Universitat de Barcelona. He is an adjunct professor at Universitat Oberta de Catalunya, Aalborg University, and Dalhousie University. He has been visiting professor at TU Delft and Aalborg Universities. He is a member of the Visual and Computational Learning consolidated research group of Catalonia. He is also a member of the Computer Vision Center at Campus UAB. He is Editor-in-Chief of American Journal of Intelligent Systems and editorial board member of more than 5 international journals. He is advisor, director, and vice-president of ChaLearn Challenges in Machine Learning, leading ChaLearn Looking at People events. He is co-founder of PhysicalTech and Care Respite companies. He is also member of the AERFAI Spanish Association on Pattern Recognition, ACIA Catalan Association of Artificial Intelligence, and he is vice-chair of IAPR TC-12: Multimedia and visual information systems. He has different patents and registered models. He has published more than 150 research papers and participated in the organization of scientific events, including CCIA04, CCIA14, ICCV11, AMDO2016, FG2017, and workshops at ICCV11, ICMI13, ECCV14, CVPR15, ICCV15, CVPR16, ECCV16, ICPR16, NIPS16. He has been guest editor at JMLR, TPAMI, IJCV, TAC, and Neural Comp. and App. He has been area chair at WACV16, NIPS16, and FG17. His research interests include, between others, statistical pattern recognition, visual object recognition, and HCI systems, with special interest in human pose recovery and behavior analysis from multi-modal data.

Jacob D Abernethy (University of Michigan)

More from the Same Authors