The Effects of Competition and Regulation on Error Inequality in Data-driven Markets

Much work has documented instances of unfairness in deployed machine learning models, and significant effort has been dedicated to creating algorithms that take into account issues of fairness. Our work highlight an important but understudied source of unfairness: market forces that drive differing amounts of firm investment in data across populations.We develop a high-level framework, based on insights from learning theory and industrial organization, to study this phenomenon. In a simple model of this type of data-driven market, we first show that a monopoly will invest unequally in the groups. There are two natural avenues for preventing this disparate impact: promoting competition and regulating firm behavior. We show first that competition, under many natural models, does not eliminate incentives to invest unequally, and can even exacerbate it. We then consider two avenues for regulating the monopoly - requiring the monopoly to ensure that each group’s error rates are low, or forcing each group’s error rates to be similar to each other - and quantify the price of fairness (and who pays it). These models imply that mitigating fairness concerns may require policy-driven solutions, and not only technological ones.

Speaker bio: Hadi Elzayn is a 4th year PhD Candidate in Applied Math and Computational Science at the University of Pennsylvania, advised by Michael Kearns. He is interested in the intersection of computer science and economics, and in the particular topics of how algorithmic learning interacts with social concerns like fairness, privacy, and markets (and how to design algorithms respecting those concerns). He received his BA from Columbia University in Mathematics and Economics. He has interned at Microsoft Research, and previously worked at the consulting firm TGG.