Skip to yearly menu bar Skip to main content

Workshop: Workshop on Human and Machine Decisions

Trucks Don’t Mean Trump: Diagnosing Human Error in Image Analysis

J.D. Zamfirescu-Pereira · Jerry Chen · Emily Wen · Allison Koenecke · Nikhil Garg · Emma Pierson


Humans often use images to make high-stakes decisions. We propose a machine learning approach to analyze the ways in which they err in doing so, leveraging a unique dataset of 16,135,392 human predictions of whether a neighborhood voted for Donald Trump or Joe Biden in the 2020 US election, based on a Google Street View image. We show that by training a machine learning estimator of the Bayes optimal decision for each image, we can provide an actionable decomposition of human error into bias, variance, and noise terms and identify specific features (like pickup trucks) which lead humans astray.

Chat is not available.