Timezone: »

 
Mix-MaxEnt: Improving Accuracy and Uncertainty Estimates of Deterministic Neural Networks
Francesco Pinto · Harry Yang · Ser Nam Lim · Philip Torr · Puneet Dokania

@
in Distribution shifts: connecting methods and applications (DistShift) »
Event URL: https://openreview.net/forum?id=hlVgM8XcssV »

We propose an extremely simple approach to regularize a single deterministic neural network to obtain improved accuracy and reliable uncertainty estimates. Our approach, on top of the cross-entropy loss, simply puts an entropy maximization regularizer corresponding to the predictive distribution in the regions of the embedding space between the class clusters. This is achieved by synthetically generating between-cluster samples via the convex combination of two images from {\em different} classes and maximizing the entropy on these samples. Such a data-dependent regularization guides the maximum likelihood estimation to prefer a solution that (1) maps out-of-distribution samples to high entropy regions (creating an entropy barrier); and (2) is more robust to the superficial input perturbations.We empirically demonstrate that Mix-MaxEnt consistently provides much improved classification accuracy, better calibrated probabilities for in-distribution data, and reliable uncertainty estimates when exposed to situations involving domain-shift and out-of-distribution samples.

Author Information

Francesco Pinto (University of Oxford)
Harry Yang (Facebook)
Ser Nam Lim (Facebook AI)
Philip Torr (University of Oxford)
Puneet Dokania (University of Oxford)

More from the Same Authors