Timezone: »
Machine Learning competitions such as the Netflix Prize have proven reasonably successful as a method of “crowdsourcing” prediction tasks. But these compe- titions have a number of weaknesses, particularly in the incentive structure they create for the participants. We propose a new approach, called a Crowdsourced Learning Mechanism, in which participants collaboratively “learn” a hypothesis for a given prediction task. The approach draws heavily from the concept of a prediction market, where traders bet on the likelihood of a future event. In our framework, the mechanism continues to publish the current hypothesis, and par- ticipants can modify this hypothesis by wagering on an update. The critical in- centive property is that a participant will profit an amount that scales according to how much her update improves performance on a released test set.
Author Information
Jacob D Abernethy (University of Michigan)
Rafael Frongillo (University of Colorado Boulder)
Related Events (a corresponding poster, oral, or spotlight)
-
2011 Poster: A Collaborative Mechanism for Crowdsourcing Prediction Problems »
Wed. Dec 14th 04:45 -- 10:59 PM Room
More from the Same Authors
-
2018 Workshop: CiML 2018 - Machine Learning competitions "in the wild": Playing in the real world or in real time »
Isabelle Guyon · Evelyne Viegas · Sergio Escalera · Jacob D Abernethy -
2018 : Building Algorithms by Playing Games »
Jacob D Abernethy -
2017 Workshop: Machine Learning Challenges as a Research Tool »
Isabelle Guyon · Evelyne Viegas · Sergio Escalera · Jacob D Abernethy -
2017 Poster: On Frank-Wolfe and Equilibrium Computation »
Jacob D Abernethy · Jun-Kun Wang -
2017 Spotlight: On Frank-Wolfe and Equilibrium Computation »
Jacob D Abernethy · Jun-Kun Wang -
2016 Workshop: Crowdsourcing and Machine Learning »
Adish Singla · Rafael Frongillo · Matteo Venanzi -
2016 Poster: Eliciting Categorical Data for Optimal Aggregation »
Chien-Ju Ho · Rafael Frongillo · Yiling Chen -
2016 Poster: Threshold Bandits, With and Without Censored Feedback »
Jacob D Abernethy · Kareem Amin · Ruihao Zhu -
2015 Poster: Fighting Bandits with a New Kind of Smoothness »
Jacob D Abernethy · Chansoo Lee · Ambuj Tewari -
2015 Poster: A Market Framework for Eliciting Private Data »
Bo Waggoner · Rafael Frongillo · Jacob D Abernethy -
2015 Poster: On Elicitation Complexity »
Rafael Frongillo · Ian Kash -
2015 Poster: Convergence Analysis of Prediction Markets via Randomized Subspace Descent »
Rafael Frongillo · Mark Reid -
2014 Workshop: NIPS’14 Workshop on Crowdsourcing and Machine Learning »
David Parkes · Denny Zhou · Chien-Ju Ho · Nihar Bhadresh Shah · Adish Singla · Jared Heyman · Edwin Simpson · Andreas Krause · Rafael Frongillo · Jennifer Wortman Vaughan · Panagiotis Papadimitriou · Damien Peters -
2014 Workshop: NIPS Workshop on Transactional Machine Learning and E-Commerce »
David Parkes · David H Wolpert · Jennifer Wortman Vaughan · Jacob D Abernethy · Amos Storkey · Mark Reid · Ping Jin · Nihar Bhadresh Shah · Mehryar Mohri · Luis E Ortiz · Robin Hanson · Aaron Roth · Satyen Kale · Sebastien Lahaie -
2013 Poster: Minimax Optimal Algorithms for Unconstrained Linear Optimization »
Brendan McMahan · Jacob D Abernethy -
2013 Poster: Adaptive Market Making via Online Learning »
Jacob D Abernethy · Satyen Kale -
2013 Poster: How to Hedge an Option Against an Adversary: Black-Scholes Pricing is Minimax Optimal »
Jacob D Abernethy · Peter Bartlett · Rafael Frongillo · Andre Wibisono -
2013 Spotlight: How to Hedge an Option Against an Adversary: Black-Scholes Pricing is Minimax Optimal »
Jacob D Abernethy · Peter Bartlett · Rafael Frongillo · Andre Wibisono -
2013 Oral: Adaptive Market Making via Online Learning »
Jacob D Abernethy · Satyen Kale -
2012 Poster: Interpreting prediction markets: a stochastic approach »
Nicolás Della Penna · Mark Reid · Rafael Frongillo -
2010 Poster: Repeated Games against Budgeted Adversaries »
Jacob D Abernethy · Manfred K. Warmuth