Improving class and group imbalanced classification with uncertainty-based active learning

Recent experimental and theoretical analyses have revealed thatuncertainty-based active learning algorithms (U-AL) are often not able toimprove the average accuracy compared to even the simple baseline of passivelearning (PL). However, we show in this work that U-AL is a competitivemethod in problems with severe data imbalance, when instead of the\emph{average} accuracy, the focus is the \emph{worst-subpopulation} accuracy.We show in extensive experiments that U-AL outperforms algorithms thatexplicitly aim to improve worst-subpopulation performance such as reweighting.We provide insights that explain the good performance of U-AL and show atheoretical result that is supported by our experimental observations.