Spotlight
PAC-Bayes-Empirical-Bernstein Inequality
Ilya Tolstikhin · Yevgeny Seldin

Sat Dec 7th 03:30 -- 03:34 PM @ Harvey's Convention Center Floor, CC

We present PAC-Bayes-Empirical-Bernstein inequality. The inequality is based on combination of PAC-Bayesian bounding technique with Empirical Bernstein bound. It allows to take advantage of small empirical variance and is especially useful in regression. We show that when the empirical variance is significantly smaller than the empirical loss PAC-Bayes-Empirical-Bernstein inequality is significantly tighter than PAC-Bayes-kl inequality of Seeger (2002) and otherwise it is comparable. PAC-Bayes-Empirical-Bernstein inequality is an interesting example of application of PAC-Bayesian bounding technique to self-bounding functions. We provide empirical comparison of PAC-Bayes-Empirical-Bernstein inequality with PAC-Bayes-kl inequality on a synthetic example and several UCI datasets.

Author Information

Ilya Tolstikhin (Google, Brain Team, Zurich)
Yevgeny Seldin (University of Copenhagen)

More from the Same Authors