Timezone: »
Deep neural networks (DNNs) have achieved tremendous success in a variety of applications across many disciplines. Yet, their superior performance comes with the expensive cost of requiring correctly annotated large-scale datasets. Moreover, due to DNNs' rich capacity, errors in training labels can hamper performance. To combat this problem, mean absolute error (MAE) has recently been proposed as a noise-robust alternative to the commonly-used categorical cross entropy (CCE) loss. However, as we show in this paper, MAE can perform poorly with DNNs and large-scale datasets. Here, we present a theoretically grounded set of noise-robust loss functions that can be seen as a generalization of MAE and CCE. Proposed loss functions can be readily applied with any existing DNN architecture and algorithm, while yielding good performance in a wide range of noisy label scenarios. We report results from experiments conducted with CIFAR-10, CIFAR-100 and FASHION-MNIST datasets and synthetically generated noisy labels.
Author Information
Zhilu Zhang (Cornell University)
Mert Sabuncu (Cornell)
Related Events (a corresponding poster, oral, or spotlight)
-
2018 Poster: Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels »
Thu Dec 6th through Fri the 7th Room Room 517 AB
More from the Same Authors
-
2020 Poster: Self-Distillation as Instance-Specific Label Smoothing »
Zhilu Zhang · Mert Sabuncu -
2020 Poster: Neural encoding with visual attention »
Meenakshi Khosla · Gia Ngo · Keith Jamison · Amy Kuceyeski · Mert Sabuncu -
2020 Oral: Neural encoding with visual attention »
Meenakshi Khosla · Gia Ngo · Keith Jamison · Amy Kuceyeski · Mert Sabuncu -
2019 Poster: Learning Conditional Deformable Templates with Convolutional Networks »
Adrian Dalca · Marianne Rakic · John Guttag · Mert Sabuncu