Timezone: »

 
Poster
Multiclass Performance Metric Elicitation
Gaurush Hiranandani · Shant Boodaghians · Ruta Mehta · Oluwasanmi Koyejo

Wed Dec 11 10:45 AM -- 12:45 PM (PST) @ East Exhibition Hall B + C #226
in Theory -- Learning Theory »

Metric Elicitation is a principled framework for selecting the performance metric that best reflects implicit user preferences. However, available strategies have so far been limited to binary classification. In this paper, we propose novel strategies for eliciting multiclass classification performance metrics using only relative preference feedback. We also show that the strategies are robust to both finite sample and feedback noise.

Keywords: Theory -> Learning Theory · Algorithms -> Active Learning; Algorithms -> Classification; Algorithms -> Ranking and Preference Learning

Author Information

Gaurush Hiranandani (University of Illinois at Urbana-Champaign)
Shant Boodaghians (UIUC)
Ruta Mehta (UIUC)
Sanmi Koyejo (UIUC)

Sanmi (Oluwasanmi) Koyejo an Assistant Professor in the Department of Computer Science at the University of Illinois at Urbana-Champaign. Koyejo's research interests are in the development and analysis of probabilistic and statistical machine learning techniques motivated by, and applied to various modern big data problems. He is particularly interested in the analysis of large scale neuroimaging data. Koyejo completed his Ph.D in Electrical Engineering at the University of Texas at Austin advised by Joydeep Ghosh, and completed postdoctoral research at Stanford University with a focus on developing Machine learning techniques for neuroimaging data. His postdoctoral research was primarily with Russell A. Poldrack and Pradeep Ravikumar. Koyejo has been the recipient of several awards including the outstanding NCE/ECE student award, a best student paper award from the conference on uncertainty in artificial intelligence (UAI) and a trainee award from the Organization for Human Brain Mapping (OHBM).

More from the Same Authors