Timezone: »

 
Can Calibration Improve Sample Prioritization?
Ganesh Tata · Gautham Krishna Gudur · Gopinath Chennupati · Mohammad Emtiyaz Khan

@
in Has it Trained Yet? A Workshop for Algorithmic Efficiency in Practical Neural Network Training »
Event URL: https://openreview.net/forum?id=LnygZu8WJk »

Calibration can reduce overconfident predictions of deep neural networks, but can calibration also accelerate training by selecting the right samples? In this paper, we show that it can. We study the effect of popular calibration techniques in selecting better subsets of samples during training (also called sample prioritization) and observe that calibration can improve the quality of subsets, reduce the number of examples per epoch (by at least 70%), and can thereby speed up the overall training process. We further study the effect of using calibrated pre-trained models coupled with calibration during training to guide sample prioritization, which again seems to improve the quality of samples selected.

Author Information

Ganesh Tata (University of Alberta)

I am a second year Master's student at the University of Alberta. I am currently pursuing my thesis under. Prof. Nilanjan ray on Optical Character Recognition (OCR) and data subset selection.

Gautham Krishna Gudur (Ericsson)
Gautham Krishna Gudur

I am a Data Scientist at Ericsson R&D in the Global AI Accelerator (GAIA) team working on machine intelligence and telecom. I also do independent research with a broad research theme of resource-efficient deep learning (accelerating neural network training, human-in-the-loop learning, etc.). Previously, I worked at SmartCardia - an AI-assisted wearable healthcare spin-off from EPFL.

Gopinath Chennupati (Amazon)
Mohammad Emtiyaz Khan (RIKEN)

Emtiyaz Khan (also known as Emti) is a team leader at the RIKEN center for Advanced Intelligence Project (AIP) in Tokyo where he leads the Approximate Bayesian Inference Team. He is also a visiting professor at the Tokyo University of Agriculture and Technology (TUAT). Previously, he was a postdoc and then a scientist at Ecole Polytechnique Fédérale de Lausanne (EPFL), where he also taught two large machine learning courses and received a teaching award. He finished his PhD in machine learning from University of British Columbia in 2012. The main goal of Emti’s research is to understand the principles of learning from data and use them to develop algorithms that can learn like living beings. For the past 10 years, his work has focused on developing Bayesian methods that could lead to such fundamental principles. The approximate Bayesian inference team now continues to use these principles, as well as derive new ones, to solve real-world problems.

More from the Same Authors