NIPS 2018 Competition Track

Hugo Jair Escalante, INAOE (Mexico), ChaLearn (USA), hugo.jair@gmail.com Isabelle Guyon, UPSud/INRIA Univ. Paris-Saclay, France & ChaLearn, USA, guyon@clopinet.com Daniel L. Silver, Acadia University, Canada, danny.silver@acadiau.ca Evelyne Viegas, Microsoft, USA, evelynev@microsoft.com Yuqiang Chen, 4Paradigm Inc., China, chenyuqiang@4paradigm.com Qiang Yang, 4Paradigm Inc., China,qyang@cse.ust.hk

Webpage: https://www.4paradigm.com/competition/nips2018

 

Adversarial Vision Challenge

Competition summary: This challenge is designed to facilitate measurable progress towards robust machine vision models and more generally applicable adversarial attacks. Modern machine vision algorithms are extremely susceptible to small and almost imperceptible perturbations of their inputs (so-called adversarial examples). This property reveals an astonishing difference in the information processing of humans and machines and raises security concerns for many deployed machine vision systems like autonomous cars. Improving the robustness of vision algorithms is thus important to close the gap between human and machine perception and to enable safety-critical applications. In a robust network no attack should be able to find imperceptible adversarial perturbations. We thus propose to facilitate an open competition between neural networks and a large variety of strong attacks, including ones that did not exist at the time when the networks have been proposed. To this end the competition has one track for robust vision models as well as one track for targeted and one for untargeted adversarial attacks. Submitted models and attacks are continuously pitted against each other on an image classification task. Attacks are able to observe the decision of models on a restricted number of self-defined inputs in order to craft model-specific minimal adversarial examples.

Organizers:

Wieland Brendel, University of T\xfcbingen, wieland@bethgelab.org
Jonas Rauber, University of T\xfcbingen, jonas@bethgelab.org
Alexey Kurakin, Google Brain, kurakin@google.com
Nicolas Papernot, Pennsylvania State University & Google Brain,ngp5056@cse.psu.edu
Behar Veliqi,University of T\xfcbingen, behar.veliqi@bethgelab.org
Marcel Salathe, Ecole Polytechnique Federale de Lausanne, marcel.salathe@epfl.ch
Sharada P. Mohanty, Ecole Polytechnique Federale de Lausanne, sharada.mohanty@epfl.ch
Matthias Bethge, University of T\xfcbingen,matthias@bethgelab.org

Webpage: https://www.crowdai.org/challenges/adversarial-vision-challenge

 

The Conversational Intelligence Challenge 2 (ConvAI2)

Competition summary: There are currently few datasets appropriate for training and evaluating models for non-goal-oriented dialogue systems (chatbots); and equally problematic, there is currently no standard procedure for evaluating such models. Our competition aims to establish a concrete scenario for testing chatbots that engage humans, and become a standard evaluation tool in order to make such systems directly comparable.This is the second Conversational Intelligence (ConvAI) Challenge. This year we introduce several improvements: a) providing a dataset from the beginning, Persona-Chat, b) making the conversations more engaging for humans, c) simpler evaluation process (automatic evaluation, followed then by human evaluation). Persona-Chat isdesigned to facilitate research into alleviating some of the issues that traditional chit-chat models face. Thetraining set consists of conversations between crowdworkers who were randomly paired and asked to act the part of a given provided persona (randomly assigned, and created by another set of crowdworkers). The paired workers were asked to chat naturally and to get to know each other during the conversation. This produces interesting and engaging conversations that learning agents can try to mimic. Models are thus trained to both ask and answer questions about personal topics, and the resulting dialogue can take account of the personas of the speaking partners. Competitors\' models will then be compared in three ways: (i) automated evaluation metrics on a new test set hidden from the competitors; (ii) evaluation on Amazon Mechanical Turk; and (iii) `wild\' live evaluation by volunteers having conversations with the bots. The winning dialogue systems will be chosen based on these scores.

Organizers:

Webpage: http://convai.io/

 

Tracking Machine Learning Challenge

Competition summary: In the footstep of the Higgs (https://www.kaggle.com/c/higgs-boson) and the flavor of physics (https://www.kaggle.com/c/flavours-of-physics) challenges, data science is being asked for provide novel ideas on how to make science advance. Particle track reconstruction is at the heart of the data processing of the experiments at CERN, and a challenging computational exercise. Contrary to first impression, clustering hundreds of thousands of sparsely 3D points into helicoidal tracks of 10-15 points is non-trivial due to combinatorial explosion during particle following. In order to fully extract the potential of collider data, and enable future scientific discoveries you will have to overcome this throughput oriented challenge and provide solutions that ran within seconds on hundreds of thousands of points. This truly unique challenge will require all your creativity and computing skills to master. In addition, the submissions will be evaluated by a jury (composed of computer scientists and High Energy Physics tracking experts) to highlight the contributions most promising to the field. A special prize (NVidia V100) from our sponsor will be attributed. Invitation to NIPS 2018 and a grand finale workshop at CERN in spring 2019 for winners and jury\u2019s pick.

Organizers:

David Rousseau, LAL, rousseau@lal.in2p3.fr
Sabrina Amrouche, UNIGE, c.amrouche@cern.ch
Paolo Calafiura, LBNL, pcalafiura@lbl.gov
Steve Farrell, LBNL, sfarrell@lbl.gov
Cecile Germain, UPsud & INRIA, cecile.germain@lri.fr
Vladimir Gligorov, LPNHE, vgligoro@lpnhe.in2p3.fr
Tobias Golling, UNIGE, tobias.golling@unige.ch
Heather Gray, LBNL, hgray@lbl.gov
Isabelle Guyon, UPsud & INRIA, Chalearn, guyon@clopinet.com
Mikhail Hushchyn, NRU HSE, mikhail.hushchyn@cern.ch
Vincenzo Innocente, CERN, vincenzo.innocente@cern.ch
Moritz Kiehn, UNIGE, msmk@cern.ch
Andreas Salzburger, CERN, andreas.salzburger@cern.ch
Andrey Ustyuzhanin, NRU HSE, andrey.ustyuzhanin@cern.ch
Jean-Roch Vlimant, Caltech, jvlimant@caltech.edu
Yetkin Yilmaz, LAL, yetkinyilmaz@gmail.com

Webpage: https://sites.google.com/site/trackmlparticle/

 

Pommerman

Competition summary: Train a team of communicative agents to play Bomberman. Compete against other teams.

Organizers:

 

InclusiveImages: A challenge of distributional skew, side information, and global inclusion

Competition summary: Questions surrounding machine learning fairness and inclusivity have attracted heightened attention in recent years, leading to a rapid emergence of a full area of research within the field of machine learning. To provide additional empirical grounding and a venue for head-to-head comparison of new methods, the InclusiveImages competition encourages researchers to develop modeling techniques that reduce the biases that may be encoded in large data sets. In particular, this competition is focused on the challenge of geographic skew encountered when the geographic distribution of training images does not fully represent levels of diversity encountered at test or inference time.

Organizers:

James Atwood
Eric Breck
Yoni Halpern
D. Sculley
Erica Greene
Peggy Chi
Anurag Batra
Contact: inclusive-images-nips@google.com

Webpage: https://sites.google.com/view/inclusiveimages/

 

The AI Driving Olympics

Competition summary: Machine Learning (ML), deep learning, and deep reinforcement learning have shown remarkable success on a variety of tasks in the very recent past. However, the ability of these methods to supersede classical approaches on physically embodied agents is still unclear. In particular, it remains to be seen whether learning-based approached can be completely trusted to control safety-critical systems such as self-driving cars. This live competition, presented by the Duckietown Foundation, is designed to explore which approaches work best for what tasks and subtasks in a complex robotic system. The participants will need to design algorithms that implement either part or all of the management and navigation required for a fleet of self-driving miniature taxis. There will be a set of different trials that correspond to progressively more sophisticated behaviors for the cars. These vary in complexity, from the reactive task of lane following to more complex and ccognitive behaviors, such as obstacle avoidance, point-to-point navigation, and finally coordinating a vehicle fleet while adhering to the entire set of the \u201crules of the road\u201d. We will provide baseline solutions for the tasks based on conventional autonomy architectures; the participants will be free to replace any or all of the components with custom learning-based solutions.The competition will be live at NIPS, but participants will not need to be physically present\u2014they will just need to send their source code packaged as a Docker image. There will be qualifying rounds in simulation and we will make available the use of \u201crobotariums,\u201d which are facilities that allow remote experimentation in a reproducible setting.

Organizers:

Andrea Censi, nuTonomy and ETH Z\xfcrich, acensi@idsc.mavt.ethz.ch
Liam Paull, Universitie de Montreal, paulll@iro.umontreal.ca
Jacopo Tani, ETH Z\xfcrich, tanij@ethz.ch
Scott Livingston, q@rerobots.net
Julian Zilly, ETH Z\xfcrich, jzilly@ethz.ch
Ruslan Hristov, nuTonomy, rusi@nutonomy.com
Oscar Beijbom, nuTonomy, oscar@nutonomy.com
Eryk Nice, nuTonomy, eryk.nice@nutonomy.com
Sunil Mallya, Amazon, smallya@amazon.com
Justin De Castri, Amazon, decastri@amazon.com
Hsueh-Cheng (Nick) Wang, National Chiao Tung University, hchengwang@gmail.com
Qing-Shan Jia, Tsinghua, jiaqs@tsinghua.edu.cn
Tao Zhang, Tsinghua , taozhang@tsinghua.edu.cn
Stefano Soatto, UCLA and Amazon, soattos@amazon.com
Magnus Egerstedt, Georgia Tech, magnus.egerstedt@ece.gatech.edu
Yoshua Bengio, Universit\xe9 de Montr\xe9al, yoshua.bengio@umontreal.ca
Emilio Frazzoli, ETH Z\xfcrich and nuTonomy, emilio.frazzoli@idsc.mavt.ethz.ch

Webpage: https://AI-DO.duckietown.org

 

AI for prosthetics

Competition summary: Recent advancements in material science and device technology have increased interest in creating prosthetics for improving human movement. Designing these devices, however, is difficult as it is costly and time-consuming to iterate through many designs. In this challenge, we explore using reinforcement learning techniques to train realistic, biomechanical models and approximate movement patterns of a patient with a prosthetic leg. Successful models will be key to better understand the human-prosthesis interaction, which will help to accelerate development of this field.

Organizers:

\u0141ukasz Kidzi\u0144ski, Stanford, lukasz.kidzinski@stanford.edu
Carmichael Ong, Stanford, ongcf@stanford.edu
Mohanty Sharada, EPFL, sharada.mohanty@epfl.ch
Jennifer Hicks, Stanford, jenhicks@stanford.edu
Joy Ku, Stanford, Stanford, joyku@stanford.edu
Sean Carroll, EPFL, sean.carroll@epfl.ch
Sergey Levine, UC Berkeley, svlevine@eecs.berkeley.edu
Marcel Salath\xe9, EPFL, marcel.salathe@epfl.ch
Scott Delp, Stanford, delp@stanford.edu

Webpage: https://www.crowdai.org/challenges/nips-2018-ai-for-prosthetics-challenge

---

'