Poster
in
Affinity Workshop: Black in AI
Predictive Multiplicity in Probabilistic Classification
Jamelle Watson-Daniels · David Parkes · Berk Ustun
Keywords: [ machine learning ]
There may exist multiple models that perform almost equally well for any given prediction task. We examine how predictions change across these competing models. In particular, we study predictive multiplicity -- in probabilistic classification. We formally define measures for our setting and develop optimization-based methods to compute these measures for convex problems. We apply our methodology to gain insight into why predictive multiplicity arises. We demonstrate the incidence and prevalence of predictive multiplicity in real-world risk assessment tasks. Our results emphasize the need to report multiplicity more widely.