Challenges in Machine Learning (CiML 2015): "Open Innovation" and "Coopetitions"

Challenges in Machine Learning have proven to be efficient and cost-effective ways to quickly bring to industry solutions that may have been confined to research. In addition, the playful nature of challenges naturally attracts students, making challenge a great teaching resource. Challenge participants range from undergraduate students to retirees, joining forces in a rewarding environment allowing them to learn, perform research, and demonstrate excellence. Therefore challenges can be used as a means of directing research, advancing the state-of-the-art or venturing in completely new domains.

Because challenges have become stream line in the execution of Machine Learning projects, it has become increasingly important to regularly bring together workshop organizers, platform providers, and participants to discuss best practices in challenge organization and new methods and application opportunities to design high impact challenges. Following the success of last year's workshop (http://ciml.chalearn.org/), in which a fruitful exchange led to many innovations, we propose to reconvene and discuss the new avenues that have been explored and lay the basis for further developments. We are particularly interested in following progresses made in two conceptually important directions:
1) Open innovation: Organization of contests in which data are made available and the contestants must both formalize and solve a problem (with some constraints), leaving more freedom to creativity, while giving more difficulty to the organizers to objectively assess the results.
2) Coopetitions: Organization of contests encouraging both collaboration and competition, in an effort to make possible the contributions of many towards a the grand goal of solving the overall problem; this poses to the organizers the problem of rewarding partial contributions.

We also want to closely follow more technical, albeit important aspects:
3) Platforms: New developments including "code submission" (platforms and protocols permitting code submission, as opposed to result submission, allowing fairer standardized comparisons in terms of hardware utilization and easier reproducibility) and " worksheets" or "scripts" facilitating code sharing.
4) Sharing, dissemination, and recognition: Facilitate sharing resources, including data, means of data collection and annotation, challenge announcements, best practices, challenge templates, publication channels, etc.; creation of awards to recognize academic services rendered by the various actors of challenge organization.