Timezone: »

 
Poster
On Optimal Generalizability in Parametric Learning
Ahmad Beirami · Meisam Razaviyayn · Shahin Shahrampour · Vahid Tarokh

Tue Dec 05 06:30 PM -- 10:30 PM (PST) @ Pacific Ballroom #18
in Posters Tue »

We consider the parametric learning problem, where the objective of the learner is determined by a parametric loss function. Employing empirical risk minimization with possibly regularization, the inferred parameter vector will be biased toward the training samples. Such bias is measured by the cross validation procedure in practice where the data set is partitioned into a training set used for training and a validation set, which is not used in training and is left to measure the out-of-sample performance. A classical cross validation strategy is the leave-one-out cross validation (LOOCV) where one sample is left out for validation and training is done on the rest of the samples that are presented to the learner, and this process is repeated on all of the samples. LOOCV is rarely used in practice due to the high computational complexity. In this paper, we first develop a computationally efficient approximate LOOCV (ALOOCV) and provide theoretical guarantees for its performance. Then we use ALOOCV to provide an optimization algorithm for finding the regularizer in the empirical risk minimization framework. In our numerical experiments, we illustrate the accuracy and efficiency of ALOOCV as well as our proposed framework for the optimization of the regularizer.

Keywords: Information Theory · Convex Optimization · Model Selection and Structure Learning · Regularization
[ Paper] 

Author Information

Ahmad Beirami (Harvard University & MIT)

Ahmad Beirami received the B.Sc. degree in electrical engineering from Sharif University of Technology, Tehran, Iran, in 2007 and the M.Sc. and Ph.D. degrees in electrical and computer engineering from the Georgia Institute of Technology, Atlanta, GA, USA, in 2011 and 2014, respectively. He is currently a postdoctoral fellow with the school of engineering and applied sciences at Harvard University, and also with electrical engineering and computer science department at MIT. Previously, he was a postdoctoral associate at Duke University. His research interests broadly include information theory, statistics, machine learning, and networks. He is the recipient of the 2013-2014 School of ECE Graduate Research Excellence Award and the 2015 Sigma Xi Best Ph.D. Thesis Award from the Georgia Institute of Technology.

Meisam Razaviyayn (University of Southern California)
Shahin Shahrampour (Harvard University)
Vahid Tarokh (Harvard University)

More from the Same Authors