Timezone: »
Most PAC-Bayesian bounds hold in the batch learning setting where data is collected at once, prior to inference or prediction. This somewhat departs from many contemporary learning problems where data streams are collected and the algorithms must dynamically adjust. We prove new PAC-Bayesian bounds in this online learning framework, leveraging an updated definition of regret, and we revisit classical PAC-Bayesian results with a batch-to-online conversion, extending their remit to the case of dependent data. Our results hold for bounded losses, potentially \emph{non-convex}, paving the way to promising developments in online learning.
Author Information
Maxime Haddouche (INRIA)
Benjamin Guedj (Inria & University College London)
Benjamin Guedj is a tenured research scientist at Inria since 2014, affiliated to the Lille - Nord Europe research centre in France. He is also affiliated with the mathematics department of the University of Lille. Since 2018, he is a Principal Research Fellow at the Centre for Artificial Intelligence and Department of Computer Science at University College London. He is also a visiting researcher at The Alan Turing Institute. Since 2020, he is the founder and scientific director of The Inria London Programme, a strategic partnership between Inria and UCL as part of a France-UK scientific initiative. He obtained his Ph.D. in mathematics in 2013 from UPMC (Université Pierre & Marie Curie, France) under the supervision of Gérard Biau and Éric Moulines. Prior to that, he was a research assistant at DTU Compute (Denmark). His main line of research is in statistical machine learning, both from theoretical and algorithmic perspectives. He is primarily interested in the design, analysis and implementation of statistical machine learning methods for high dimensional problems, mainly using the PAC-Bayesian theory.
More from the Same Authors
-
2021 : Progress in Self-Certified Neural Networks »
Maria Perez-Ortiz · Omar Rivasplata · Emilio Parrado-Hernández · Benjamin Guedj · John Shawe-Taylor -
2022 Poster: KSD Aggregated Goodness-of-fit Test »
Antonin Schrab · Benjamin Guedj · Arthur Gretton -
2022 Poster: Efficient Aggregated Kernel Tests using Incomplete $U$-statistics »
Antonin Schrab · Ilmun Kim · Benjamin Guedj · Arthur Gretton -
2022 Poster: On Margins and Generalisation for Voting Classifiers »
Felix Biggs · Valentina Zantedeschi · Benjamin Guedj -
2021 Poster: Learning Stochastic Majority Votes by Minimizing a PAC-Bayes Generalization Bound »
Valentina Zantedeschi · Paul Viallard · Emilie Morvant · Rémi Emonet · Amaury Habrard · Pascal Germain · Benjamin Guedj -
2020 Poster: PAC-Bayesian Bound for the Conditional Value at Risk »
Zakaria Mhammedi · Benjamin Guedj · Robert Williamson -
2020 Spotlight: PAC-Bayesian Bound for the Conditional Value at Risk »
Zakaria Mhammedi · Benjamin Guedj · Robert Williamson -
2019 Poster: PAC-Bayes Un-Expected Bernstein Inequality »
Zakaria Mhammedi · Peter Grünwald · Benjamin Guedj -
2019 Poster: Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks »
Gaël Letarte · Pascal Germain · Benjamin Guedj · Francois Laviolette -
2017 : Concluding remarks »
Francis Bach · Benjamin Guedj · Pascal Germain -
2017 : Overture »
Benjamin Guedj · Francis Bach · Pascal Germain -
2017 Workshop: (Almost) 50 shades of Bayesian Learning: PAC-Bayesian trends and insights »
Benjamin Guedj · Pascal Germain · Francis Bach