Timezone: »
The mathematical analysis and understanding of rank data has been a fascinating topic for centuries, and has been investigated in disciplines as wide-ranging as social choice/voting theory, decision theory, probability, statistics, and combinatorics. In modern times, huge amounts of data are generated in the form of rankings on a daily basis: restaurant ratings, product ratings/comparisons, employer ratings, hospital rankings, doctor rankings, and an endless variety of rankings from committee deliberations (including, for example, deliberations of conference program committees such as NIPS!). These applications have led to several new trends and challenges: for example, one must frequently deal with very large numbers of candidates/alternatives to be ranked, with partial or missing ranking information, with noisy ranking information, with the need to ensure reliability and/or privacy of the rank data provided, and so on.
Given the increasing universality of settings involving large amounts of rank data and associated challenges as above, powerful computational frameworks and tools for addressing such challenges have emerged over the last few years in a variety of areas, including in particular in machine learning, operations research, and computational social choice. Despite the fact that many important practical problems in each area could benefit from the algorithmic solutions and analysis techniques developed in other areas, there has been limited interaction between these areas. Given both the increasing maturity of the research into ranking in these respective areas and the increasing range of practical ranking problems in need of better solutions, it is the aim of this workshop to bring together recent advances in analyzing rank data in machine learning, operations research, and computational social choice under one umbrella, to enable greater interaction and cross-fertilization of ideas.
A primary goal will be to discover connections between recent approaches developed for analyzing rank data in each of the three areas above. To this end, we will have invited talks by leading experts in the analysis of rank data in each area. In addition, we will include perspectives from practitioners who work with rank data in various applied domains on both the benefits and limitations of currently available solutions to the problems they encounter. In the end, we hope to both develop a shared language for the analysis and understanding of rank data in modern times, and identify important challenges that persist and could benefit from a shared understanding.
The topics of interest include:
- discrete choice modeling and revenue management
- voting and social decision making, preference elicitation
- social choice (rank aggregation) versus individual choice (recommendation systems)
- stochastic versus active sampling of preferences
- statistical/learning-theoretic guarantees
- effects of computational approximations
Author Information
Shivani Agarwal (University of Pennsylvania)
Hossein Azari Soufiani (Harvard University)
Guy Bresler (Massachusetts Institute of Technology)
Sewoong Oh (UIUC)
David Parkes (Harvard University)
David C. Parkes is Gordon McKay Professor of Computer Science in the School of Engineering and Applied Sciences at Harvard University. He was the recipient of the NSF Career Award, the Alfred P. Sloan Fellowship, the Thouron Scholarship and the Harvard University Roslyn Abramson Award for Teaching. Parkes received his Ph.D. degree in Computer and Information Science from the University of Pennsylvania in 2001, and an M.Eng. (First class) in Engineering and Computing Science from Oxford University in 1995. At Harvard, Parkes leads the EconCS group and teaches classes in artificial intelligence, optimization, and topics at the intersection between computer science and economics. Parkes has served as Program Chair of ACM EC’07 and AAMAS’08 and General Chair of ACM EC’10, served on the editorial board of Journal of Artificial Intelligence Research, and currently serves as Editor of Games and Economic Behavior and on the boards of Journal of Autonomous Agents and Multi-agent Systems and INFORMS Journal of Computing. His research interests include computational mechanism design, electronic commerce, stochastic optimization, preference elicitation, market design, bounded rationality, computational social choice, networks and incentives, multi-agent systems, crowd-sourcing and social computing.
Arun Rajkumar (Xerox Research Center, India.)
Devavrat Shah (Massachusetts Institute of Technology)
Devavrat Shah is a professor of Electrical Engineering & Computer Science and Director of Statistics and Data Science at MIT. He received PhD in Computer Science from Stanford. He received Erlang Prize from Applied Probability Society of INFORMS in 2010 and NeuIPS best paper award in 2008.
More from the Same Authors
-
2021 Spotlight: Regulating algorithmic filtering on social media »
Sarah Cen · Devavrat Shah -
2021 : Deep Reinforcement Learning Explanation via Model Transforms »
Sarah Keren · Yoav Kolumbus · Jeffrey S Rosenschein · David Parkes · Mira Finkelstein -
2021 : Regret, stability, and fairness in matching markets with bandit learners »
Sarah Cen · Devavrat Shah -
2021 : Regret, stability, and fairness in matching markets with bandit learners »
Sarah Cen · Devavrat Shah -
2022 : Predictive Multiplicity in Probabilistic Classification »
Jamelle Watson-Daniels · David Parkes · Berk Ustun -
2022 : A Causal Inference Framework for Network Interference with Panel Data »
Sarah Cen · Anish Agarwal · Christina Yu · Devavrat Shah -
2022 : On counterfactual inference with unobserved confounding »
Abhin Shah · Raaz Dwivedi · Devavrat Shah · Gregory Wornell -
2022 : Predictive Multiplicity in Probabilistic Classification »
Jamelle Watson-Daniels · David Parkes · Berk Ustun -
2022 : Learning to Mitigate AI Collusion on E-Commerce Platforms »
Eric Mibuari · Gianluca Brero · David Parkes · Nicolas Lepore -
2022 : Meta-RL for Multi-Agent RL: Learning to Adapt to Evolving Agents »
Matthias Gerstgrasser · David Parkes -
2023 Poster: Data Market Design through Deep Learning »
Sai Srivatsa Ravindranath · Yanchen Jiang · David Parkes -
2023 Poster: Deep Contract Design via Discontinuous Piecewise Affine Neural Networks »
Tonghan Wang · Paul Duetting · Dmitry Ivanov · Inbal Talgam-Cohen · David Parkes -
2023 Poster: Auditing for Human Expertise »
Rohan Alur · Loren Laine · Darrick Li · Manish Raghavan · Devavrat Shah · Dennis Shung -
2023 Poster: SAMoSSA: Multivariate Singular Spectrum Analysis with Stochastic Autoregressive Noise »
Abdullah Alomar · Munther Dahleh · Sean Mann · Devavrat Shah -
2022 Spotlight: Lightning Talks 5A-2 »
Qiang LI · Zhiwei Xu · Jia-Qi Yang · Thai Hung Le · Haoxuan Qu · Yang Li · Artyom Sorokin · Peirong Zhang · Mira Finkelstein · Nitsan levy · Chung-Yiu Yau · dapeng li · Thommen Karimpanal George · De-Chuan Zhan · Nazar Buzun · Jiajia Jiang · Li Xu · Yichuan Mo · Yujun Cai · Yuliang Liu · Leonid Pugachev · Bin Zhang · Lucy Liu · Hoi-To Wai · Liangliang Shi · Majid Abdolshah · Yoav Kolumbus · Lin Geng Foo · Junchi Yan · Mikhail Burtsev · Lianwen Jin · Yuan Zhan · Dung Nguyen · David Parkes · Yunpeng Baiia · Jun Liu · Kien Do · Guoliang Fan · Jeffrey S Rosenschein · Sunil Gupta · Sarah Keren · Svetha Venkatesh -
2022 Spotlight: Explainable Reinforcement Learning via Model Transforms »
Mira Finkelstein · Nitsan levy · Lucy Liu · Yoav Kolumbus · David Parkes · Jeffrey S Rosenschein · Sarah Keren -
2022 Poster: Explainable Reinforcement Learning via Model Transforms »
Mira Finkelstein · Nitsan levy · Lucy Liu · Yoav Kolumbus · David Parkes · Jeffrey S Rosenschein · Sarah Keren -
2022 Poster: Learning to Mitigate AI Collusion on Economic Platforms »
Gianluca Brero · Eric Mibuari · Nicolas Lepore · David Parkes -
2021 Workshop: Learning in Presence of Strategic Behavior »
Omer Ben-Porat · Nika Haghtalab · Annie Liang · Yishay Mansour · David Parkes -
2021 Poster: A Computationally Efficient Method for Learning Exponential Family Distributions »
Abhin Shah · Devavrat Shah · Gregory Wornell -
2021 Poster: Regulating algorithmic filtering on social media »
Sarah Cen · Devavrat Shah -
2021 Poster: Change Point Detection via Multivariate Singular Spectrum Analysis »
Arwa Alanqary · Abdullah Alomar · Devavrat Shah -
2021 Poster: PerSim: Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents via Personalized Simulators »
Anish Agarwal · Abdullah Alomar · Varkey Alumootil · Devavrat Shah · Dennis Shen · Zhi Xu · Cindy Yang -
2020 Workshop: Machine Learning for Economic Policy »
Stephan Zheng · Alexander Trott · Annie Liang · Jamie Morgenstern · David Parkes · Nika Haghtalab -
2020 Poster: Estimation of Skill Distribution from a Tournament »
Ali Jadbabaie · Anuran Makur · Devavrat Shah -
2020 Spotlight: Estimation of Skill Distribution from a Tournament »
Ali Jadbabaie · Anuran Makur · Devavrat Shah -
2020 Poster: From Predictions to Decisions: Using Lookahead Regularization »
Nir Rosenfeld · Anna Hilgard · Sai Srivatsa Ravindranath · David Parkes -
2020 Poster: Sample Efficient Reinforcement Learning via Low-Rank Matrix Estimation »
Devavrat Shah · Dogyoon Song · Zhi Xu · Yuzhe Yang -
2020 Demonstration: tspDB: Time Series Predict DB »
Anish Agarwal · Abdullah Alomar · Devavrat Shah -
2019 Poster: On Robustness of Principal Component Regression »
Anish Agarwal · Devavrat Shah · Dennis Shen · Dogyoon Song -
2019 Oral: On Robustness of Principal Component Regression »
Anish Agarwal · Devavrat Shah · Dennis Shen · Dogyoon Song -
2019 Poster: Finding Friend and Foe in Multi-Agent Games »
Jack Serrino · Max Kleiman-Weiner · David Parkes · Josh Tenenbaum -
2019 Spotlight: Finding Friend and Foe in Multi-Agent Games »
Jack Serrino · Max Kleiman-Weiner · David Parkes · Josh Tenenbaum -
2019 Tutorial: Synthetic Control »
Alberto Abadie · Vishal Misra · Devavrat Shah -
2018 Poster: Q-learning with Nearest Neighbors »
Devavrat Shah · Qiaomin Xie -
2017 : Optimal Economic Design through Deep Learning »
David Parkes -
2017 : New perspective from Blackwell's "comparisons of experiments" on generative adversarial networks »
Sewoong Oh -
2017 Workshop: Nearest Neighbors for Modern Applications with Massive Data: An Age-old Solution with New Challenges »
George H Chen · Devavrat Shah · Christina Lee -
2017 Poster: Optimal Sample Complexity of M-wise Data for Top-K Ranking »
Minje Jang · Sunghyun Kim · Changho Suh · Sewoong Oh -
2017 Poster: Thy Friend is My Friend: Iterative Collaborative Filtering for Sparse Matrix Estimation »
Christian Borgs · Jennifer Chayes · Christina Lee · Devavrat Shah -
2017 Poster: Multi-View Decision Processes: The Helper-AI Problem »
Christos Dimitrakakis · David Parkes · Goran Radanovic · Paul Tylkin -
2017 Poster: Estimating Mutual Information for Discrete-Continuous Mixtures »
Weihao Gao · Sreeram Kannan · Sewoong Oh · Pramod Viswanath -
2017 Poster: Matrix Norm Estimation from a Few Entries »
Ashish Khetan · Sewoong Oh -
2017 Spotlight: Estimating Mutual Information for Discrete-Continuous Mixtures »
Weihao Gao · Sreeram Kannan · Sewoong Oh · Pramod Viswanath -
2017 Spotlight: Matrix Norm Estimation from a Few Entries »
Ashish Khetan · Sewoong Oh -
2017 Poster: Discovering Potential Correlations via Hypercontractivity »
Hyeji Kim · Weihao Gao · Sreeram Kannan · Sewoong Oh · Pramod Viswanath -
2016 Poster: Long-term Causal Effects via Behavioral Game Theory »
Panagiotis Toulis · David Parkes -
2016 Poster: Breaking the Bandwidth Barrier: Geometrical Adaptive Entropy Estimation »
Weihao Gao · Sewoong Oh · Pramod Viswanath -
2016 Poster: Computational and Statistical Tradeoffs in Learning to Rank »
Ashish Khetan · Sewoong Oh -
2016 Poster: Blind Regression: Nonparametric Regression for Latent Variable Models via Collaborative Filtering »
Dogyoon Song · Christina Lee · Yihua Li · Devavrat Shah -
2016 Poster: Achieving budget-optimality with adaptive schemes in crowdsourcing »
Ashish Khetan · Sewoong Oh -
2016 Poster: Dueling Bandits: Beyond Condorcet Winners to General Tournament Solutions »
Siddartha Ramamohan · Arun Rajkumar · Shivani Agarwal · Shivani Agarwal -
2015 Workshop: Non-convex Optimization for Machine Learning: Theory and Practice »
Anima Anandkumar · Niranjan Uma Naresh · Kamalika Chaudhuri · Percy Liang · Sewoong Oh -
2015 Poster: Secure Multi-party Differential Privacy »
Peter Kairouz · Sewoong Oh · Pramod Viswanath -
2015 Poster: Learnability of Influence in Networks »
Harikrishna Narasimhan · David Parkes · Yaron Singer -
2015 Poster: Collaboratively Learning Preferences from Ordinal Data »
Sewoong Oh · Kiran Thekumparampil · Jiaming Xu -
2014 Workshop: NIPS’14 Workshop on Crowdsourcing and Machine Learning »
David Parkes · Denny Zhou · Chien-Ju Ho · Nihar Bhadresh Shah · Adish Singla · Jared Heyman · Edwin Simpson · Andreas Krause · Rafael Frongillo · Jennifer Wortman Vaughan · Panagiotis Papadimitriou · Damien Peters -
2014 Workshop: NIPS Workshop on Transactional Machine Learning and E-Commerce »
David Parkes · David H Wolpert · Jennifer Wortman Vaughan · Jacob D Abernethy · Amos Storkey · Mark Reid · Ping Jin · Nihar Bhadresh Shah · Mehryar Mohri · Luis E Ortiz · Robin Hanson · Aaron Roth · Satyen Kale · Sebastien Lahaie -
2014 Poster: A Statistical Decision-Theoretic Framework for Social Choice »
Hossein Azari Soufiani · David Parkes · Lirong Xia -
2014 Poster: Hardness of parameter estimation in graphical models »
Guy Bresler · David Gamarnik · Devavrat Shah -
2014 Oral: A Statistical Decision-Theoretic Framework for Social Choice »
Hossein Azari Soufiani · David Parkes · Lirong Xia -
2014 Poster: On the Statistical Consistency of Plug-in Classifiers for Non-decomposable Performance Measures »
Harikrishna Narasimhan · Rohit Vaish · Shivani Agarwal -
2014 Poster: Provable Tensor Factorization with Missing Data »
Prateek Jain · Sewoong Oh -
2014 Poster: A Latent Source Model for Online Collaborative Filtering »
Guy Bresler · George H Chen · Devavrat Shah -
2014 Poster: Extremal Mechanisms for Local Differential Privacy »
Peter Kairouz · Sewoong Oh · Pramod Viswanath -
2014 Poster: Minimax-optimal Inference from Partial Rankings »
Bruce Hajek · Sewoong Oh · Jiaming Xu -
2014 Spotlight: A Latent Source Model for Online Collaborative Filtering »
Guy Bresler · George H Chen · Devavrat Shah -
2014 Poster: Learning Mixed Multinomial Logit Model from Ordinal Data »
Sewoong Oh · Devavrat Shah -
2014 Poster: Online Decision-Making in General Combinatorial Spaces »
Arun Rajkumar · Shivani Agarwal -
2014 Poster: Structure learning of antiferromagnetic Ising models »
Guy Bresler · David Gamarnik · Devavrat Shah -
2013 Workshop: Crowdsourcing: Theory, Algorithms and Applications »
Jennifer Wortman Vaughan · Greg Stoddard · Chien-Ju Ho · Adish Singla · Michael Bernstein · Devavrat Shah · Arpita Ghosh · Evgeniy Gabrilovich · Denny Zhou · Nikhil Devanur · Xi Chen · Alexander Ihler · Qiang Liu · Genevieve Patterson · Ashwinkumar Badanidiyuru Varadaraja · Hossein Azari Soufiani · Jacob Whitehill -
2013 Poster: A Latent Source Model for Nonparametric Time Series Classification »
George H Chen · Stanislav Nikolov · Devavrat Shah -
2013 Poster: Convex Calibrated Surrogates for Low-Rank Loss Matrices with Applications to Subset Ranking Losses »
Harish G Ramaswamy · Shivani Agarwal · Ambuj Tewari -
2013 Poster: On the Relationship Between Binary Classification, Bipartite Ranking, and Binary Class Probability Estimation »
Harikrishna Narasimhan · Shivani Agarwal -
2013 Spotlight: On the Relationship Between Binary Classification, Bipartite Ranking, and Binary Class Probability Estimation »
Harikrishna Narasimhan · Shivani Agarwal -
2013 Spotlight: Convex Calibrated Surrogates for Low-Rank Loss Matrices with Applications to Subset Ranking Losses »
Harish G Ramaswamy · Shivani Agarwal · Ambuj Tewari -
2013 Poster: Generalized Random Utility Models with Multiple Types »
Hossein Azari Soufiani · Hansheng Diao · Zhenyu Lai · David Parkes -
2013 Poster: Computing the Stationary Distribution Locally »
Christina Lee · Asuman Ozdaglar · Devavrat Shah -
2013 Poster: Contrastive Learning Using Spectral Methods »
James Y Zou · Daniel Hsu · David Parkes · Ryan Adams -
2013 Poster: Generalized Method-of-Moments for Rank Aggregation »
Hossein Azari Soufiani · William Z Chen · David Parkes · Lirong Xia -
2012 Poster: Iterative ranking from pair-wise comparisons »
Sahand N Negahban · Sewoong Oh · Devavrat Shah -
2012 Poster: Random Utility Theory for Social Choice: Theory and Algorithms »
Hossein Azari Soufiani · David C Parkes · Lirong Xia -
2012 Poster: Classification Calibration Dimension for General Multiclass Losses »
Harish G Ramaswamy · Shivani Agarwal -
2012 Spotlight: Iterative ranking from pair-wise comparisons »
Sahand N Negahban · Sewoong Oh · Devavrat Shah -
2012 Spotlight: Classification Calibration Dimension for General Multiclass Losses »
Harish G Ramaswamy · Shivani Agarwal -
2011 Poster: Iterative Learning for Reliable Crowdsourcing Systems »
David R Karger · Sewoong Oh · Devavrat Shah -
2011 Oral: Iterative Learning for Reliable Crowdsourcing Systems »
David R Karger · Sewoong Oh · Devavrat Shah -
2010 Invited Talk: The Interplay of Machine Learning and Mechanism Design »
David Parkes -
2009 Workshop: Advances in Ranking »
Shivani Agarwal · Chris J Burges · Yacov Crammer -
2009 Poster: A Data-Driven Approach to Modeling Choice »
Vivek Farias · Srikanth Jagabathula · Devavrat Shah -
2009 Spotlight: A Data-Driven Approach to Modeling Choice »
Vivek Farias · Srikanth Jagabathula · Devavrat Shah -
2009 Poster: Local Rules for Global MAP: When Do They Work ? »
Kyomin Jung · Pushmeet Kohli · Devavrat Shah -
2009 Poster: Matrix Completion from Noisy Entries »
Raghunandan Keshavan · Andrea Montanari · Sewoong Oh -
2008 Poster: Inferring rankings under constrained sensing »
Srikanth Jagabathula · Devavrat Shah -
2008 Oral: Inferring rankings under constrained sensing »
Srikanth Jagabathula · Devavrat Shah -
2007 Spotlight: Message Passing for Max-weight Independent Set »
Sujay Sanghavi · Devavrat Shah · Alan S Willsky -
2007 Poster: Message Passing for Max-weight Independent Set »
Sujay Sanghavi · Devavrat Shah · Alan S Willsky -
2007 Poster: Local Algorithms for Approximate Inference in Minor-Excluded Graphs »
Kyomin Jung · Devavrat Shah