Poster
On the Calibration of Multiclass Classification with Rejection
Chenri Ni · Nontawat Charoenphakdee · Junya Honda · Masashi Sugiyama

Tue Dec 10th 10:45 AM -- 12:45 PM @ East Exhibition Hall B + C #224
in Theory -- Learning Theory »

We investigate the problem of multiclass classification with rejection, where a classifier can choose not to make a prediction to avoid critical misclassification. First, we consider an approach based on simultaneous training of a classifier and a rejector, which achieves the state-of-the-art performance in the binary case. We analyze this approach for the multiclass case and derive a general condition for calibration to the Bayes-optimal solution, which suggests that calibration is hard to achieve by general loss functions unlike the binary case. Next, we consider another traditional approach based on confidence scores, in which the existing work focuses on a specific class of losses. We propose rejection criteria for more general losses for this approach and guarantee calibration to the Bayes-optimal solution. Finally, we conduct experiments to validate the relevance of our theoretical findings.

Keywords: Algorithms -> Classification · Theory -> Learning Theory

Author Information

Chenri Ni (The University of Tokyo)
Nontawat Charoenphakdee (The University of Tokyo / RIKEN)
Junya Honda (The University of Tokyo / RIKEN)
Masashi Sugiyama (RIKEN / University of Tokyo)

More from the Same Authors