Timezone: »

 
Trucks Don’t Mean Trump: Diagnosing Human Error in Image Analysis
J.D. Zamfirescu-Pereira · Jerry Chen · Emily Wen · Allison Koenecke · Nikhil Garg · Emma Pierson

@
in Workshop on Human and Machine Decisions »

Humans often use images to make high-stakes decisions. We propose a machine learning approach to analyze the ways in which they err in doing so, leveraging a unique dataset of 16,135,392 human predictions of whether a neighborhood voted for Donald Trump or Joe Biden in the 2020 US election, based on a Google Street View image. We show that by training a machine learning estimator of the Bayes optimal decision for each image, we can provide an actionable decomposition of human error into bias, variance, and noise terms and identify specific features (like pickup trucks) which lead humans astray.

Author Information

J.D. Zamfirescu-Pereira (UC Berkeley)
Jerry Chen (Stanford University)
Emily Wen (Stanford University)
Allison Koenecke (Cornell University)
Nikhil Garg (Cornell Tech)
Emma Pierson (Microsoft Research)

More from the Same Authors