Timezone: »

 
Poster
On the Complexity of Linear Prediction: Risk Bounds, Margin Bounds, and Regularization
Sham M Kakade · Karthik Sridharan · Ambuj Tewari

Mon Dec 08 08:45 PM -- 12:00 AM (PST) @
in Posters »
We provide sharp bounds for Rademacher and Gaussian complexities of (constrained) linear classes. These bounds make short work of providing a number of corollaries including: risk bounds for linear prediction (including settings where the weight vectors are constrained by either $L_2$ or $L_1$ constraints), margin bounds (including both $L_2$ and $L_1$ margins, along with more general notions based on relative entropy), a proof of the PAC-Bayes theorem, and $L_2$ covering numbers (with $L_p$ norm constraints and relative entropy constraints). In addition to providing a unified analysis, the results herein provide some of the sharpest risk and margin bounds (improving upon a number of previous results). Interestingly, our results show that the uniform convergence rates of empirical risk minimization algorithms tightly match the regret bounds of online learning algorithms for linear prediction (up to a constant factor of 2).
[ PDF] 

Author Information

Sham M Kakade (Harvard University & Amazon)
Karthik Sridharan (University of Pennsylvania)
Ambuj Tewari (University of Michigan)

More from the Same Authors