Timezone: »

 
Poster
Multiclass Performance Metric Elicitation
Gaurush Hiranandani · Shant Boodaghians · Ruta Mehta · Sanmi Koyejo

Wed Dec 11 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #226
in Theory -- Learning Theory »

Metric Elicitation is a principled framework for selecting the performance metric that best reflects implicit user preferences. However, available strategies have so far been limited to binary classification. In this paper, we propose novel strategies for eliciting multiclass classification performance metrics using only relative preference feedback. We also show that the strategies are robust to both finite sample and feedback noise.

Keywords: Learning Theory · Ranking and Preference Learning · Theory · Algorithms -> Active Learning; Algorithms -> Classification; Algorithms

Author Information

Gaurush Hiranandani (University of Illinois at Urbana-Champaign)
Shant Boodaghians (UIUC)
Ruta Mehta (UIUC)
Sanmi Koyejo (UIUC)
Sanmi Koyejo

Sanmi Koyejo an Assistant Professor in the Department of Computer Science at Stanford University. Koyejo also spends time at Google as a part of the Brain team. Koyejo's research interests are in developing the principles and practice of trustworthy machine learning. Additionally, Koyejo focuses on applications to neuroscience and healthcare. Koyejo has been the recipient of several awards, including a best paper award from the conference on uncertainty in artificial intelligence (UAI), a Skip Ellis Early Career Award, and a Sloan Fellowship. Koyejo serves as the president of the Black in AI organization.

More from the Same Authors