Timezone: »
Poster
Spectrally-normalized margin bounds for neural networks
Peter Bartlett · Dylan J Foster · Matus Telgarsky
This paper presents a margin-based multiclass generalization bound for neural networks that scales with their margin-normalized "spectral complexity": their Lipschitz constant, meaning the product of the spectral norms of the weight matrices, times a certain correction factor. This bound is empirically investigated for a standard AlexNet network trained with SGD on the MNIST and CIFAR10 datasets, with both original and random labels; the bound, the Lipschitz constants, and the excess risks are all in direct correlation, suggesting both that SGD selects predictors whose complexity scales with the difficulty of the learning task, and secondly that the presented bound is sensitive to this complexity.
Author Information
Peter Bartlett (UC Berkeley)
Dylan J Foster (Cornell University)
Matus Telgarsky (UIUC)
Related Events (a corresponding poster, oral, or spotlight)
-
2017 Spotlight: Spectrally-normalized margin bounds for neural networks »
Wed Dec 6th 07:50 -- 07:55 PM Room Hall A
More from the Same Authors
-
2020 Poster: Directional convergence and alignment in deep learning »
Ziwei Ji · Matus Telgarsky -
2020 Spotlight: Directional convergence and alignment in deep learning »
Ziwei Ji · Matus Telgarsky -
2018 Poster: Size-Noise Tradeoffs in Generative Networks »
Bolton Bailey · Matus Telgarsky -
2018 Spotlight: Size-Noise Tradeoffs in Generative Networks »
Bolton Bailey · Matus Telgarsky -
2017 Poster: Near Minimax Optimal Players for the Finite-Time 3-Expert Prediction Problem »
Yasin Abbasi Yadkori · Peter Bartlett · Victor Gabillon -
2017 Poster: Alternating minimization for dictionary learning with random initialization »
Niladri Chatterji · Peter Bartlett -
2017 Poster: Parameter-Free Online Learning via Model Selection »
Dylan J Foster · Satyen Kale · Mehryar Mohri · Karthik Sridharan -
2017 Poster: Acceleration and Averaging in Stochastic Descent Dynamics »
Walid Krichene · Peter Bartlett -
2017 Spotlight: Parameter-Free Online Learning via Model Selection »
Dylan J Foster · Satyen Kale · Mehryar Mohri · Karthik Sridharan -
2017 Spotlight: Acceleration and Averaging in Stochastic Descent Dynamics »
Walid Krichene · Peter Bartlett -
2014 Poster: Scalable Non-linear Learning with Adaptive Polynomial Expansions »
Alekh Agarwal · Alina Beygelzimer · Daniel Hsu · John Langford · Matus J Telgarsky -
2013 Poster: Moment-based Uniform Deviation Bounds for $k$-means and Friends »
Matus J Telgarsky · Sanjoy Dasgupta -
2011 Poster: The Fast Convergence of Boosting »
Matus J Telgarsky