Timezone: »

 
Oral
Retiring Adult: New Datasets for Fair Machine Learning
Frances Ding · Moritz Hardt · John Miller · Ludwig Schmidt

Fri Dec 10 04:20 PM -- 04:35 PM (PST) @
in Oral Session 5: Fairness and Privacy »

Although the fairness community has recognized the importance of data, researchers in the area primarily rely on UCI Adult when it comes to tabular data. Derived from a 1994 US Census survey, this dataset has appeared in hundreds of research papers where it served as the basis for the development and comparison of many algorithmic fairness interventions. We reconstruct a superset of the UCI Adult data from available US Census sources and reveal idiosyncrasies of the UCI Adult dataset that limit its external validity. Our primary contribution is a suite of new datasets derived from US Census surveys that extend the existing data ecosystem for research on fair machine learning. We create prediction tasks relating to income, employment, health, transportation, and housing. The data span multiple years and all states of the United States, allowing researchers to study temporal shift and geographic variation. We highlight a broad initial sweep of new empirical insights relating to trade-offs between fairness criteria, performance of algorithmic interventions, and the role of distribution shift based on our new datasets. Our findings inform ongoing debates, challenge some existing narratives, and point to future research directions.

Keywords: Machine Learning · Graph Learning · Fairness

Author Information

Frances Ding (University of California Berkeley)
Moritz Hardt (University of California, Berkeley)
John Miller (UC Berkeley)
Ludwig Schmidt (University of Washington)

Related Events (a corresponding poster, oral, or spotlight)

More from the Same Authors