Timezone: »
We investigate a stochastic counterpart of majority votes over finite ensembles of classifiers, and study its generalization properties. While our approach holds for arbitrary distributions, we instantiate it with Dirichlet distributions: this allows for a closed-form and differentiable expression for the expected risk, which then turns the generalization bound into a tractable training objective.The resulting stochastic majority vote learning algorithm achieves state-of-the-art accuracy and benefits from (non-vacuous) tight generalization bounds, in a series of numerical experiments when compared to competing algorithms which also minimize PAC-Bayes objectives -- both with uninformed (data-independent) and informed (data-dependent) priors.
Author Information
Valentina Zantedeschi (INRIA & UCL)
Paul Viallard (University of Saint-Etienne, Lab Hubert Curien)
Emilie Morvant (LaHC, University of Saint-Etienne)
Rémi Emonet (Hubert Curien Lab.)
Amaury Habrard (University of Saint-Etienne, Lab. H Curien, France)
Pascal Germain (Université Laval)
Benjamin Guedj (Inria & University College London)
Benjamin Guedj is a tenured research scientist at Inria since 2014, affiliated to the Lille - Nord Europe research centre in France. He is also affiliated with the mathematics department of the University of Lille. Since 2018, he is a Principal Research Fellow at the Centre for Artificial Intelligence and Department of Computer Science at University College London. He is also a visiting researcher at The Alan Turing Institute. Since 2020, he is the founder and scientific director of The Inria London Programme, a strategic partnership between Inria and UCL as part of a France-UK scientific initiative. He obtained his Ph.D. in mathematics in 2013 from UPMC (Université Pierre & Marie Curie, France) under the supervision of Gérard Biau and Éric Moulines. Prior to that, he was a research assistant at DTU Compute (Denmark). His main line of research is in statistical machine learning, both from theoretical and algorithmic perspectives. He is primarily interested in the design, analysis and implementation of statistical machine learning methods for high dimensional problems, mainly using the PAC-Bayesian theory.
More from the Same Authors
-
2021 : Progress in Self-Certified Neural Networks »
Maria Perez-Ortiz · Omar Rivasplata · Emilio Parrado-Hernández · Benjamin Guedj · John Shawe-Taylor -
2022 : Discrete Learning Of DAGs Via Backpropagation »
Andrew Wren · Pasquale Minervini · Luca Franceschi · Valentina Zantedeschi -
2022 : Discrete Learning Of DAGs Via Backpropagation »
Andrew Wren · Pasquale Minervini · Luca Franceschi · Valentina Zantedeschi -
2023 Poster: Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theory »
Sokhna Diarra Mbacke · Florence Clerc · Pascal Germain -
2023 Poster: Learning via Wasserstein-Based High Probability Generalization Bounds »
Paul Viallard · Maxime Haddouche · Umut Simsekli · Benjamin Guedj -
2022 : Discrete Learning Of DAGs Via Backpropagation »
Andrew Wren · Pasquale Minervini · Luca Franceschi · Valentina Zantedeschi -
2022 Poster: KSD Aggregated Goodness-of-fit Test »
Antonin Schrab · Benjamin Guedj · Arthur Gretton -
2022 Poster: Efficient Aggregated Kernel Tests using Incomplete $U$-statistics »
Antonin Schrab · Ilmun Kim · Benjamin Guedj · Arthur Gretton -
2022 Poster: On Margins and Generalisation for Voting Classifiers »
Felix Biggs · Valentina Zantedeschi · Benjamin Guedj -
2022 Poster: Online PAC-Bayes Learning »
Maxime Haddouche · Benjamin Guedj -
2021 Poster: A PAC-Bayes Analysis of Adversarial Robustness »
Paul Viallard · Eric Guillaume VIDOT · Amaury Habrard · Emilie Morvant -
2020 : Towards Data-Driven Physics-Informed Global Precipitation Forecasting from Satellite Imagery »
Valentina Zantedeschi · Valentina Zantedeschi -
2020 Poster: PAC-Bayesian Bound for the Conditional Value at Risk »
Zakaria Mhammedi · Benjamin Guedj · Robert Williamson -
2020 Spotlight: PAC-Bayesian Bound for the Conditional Value at Risk »
Zakaria Mhammedi · Benjamin Guedj · Robert Williamson -
2019 : Break / Poster Session 1 »
Antonia Marcu · Yao-Yuan Yang · Pascale Gourdeau · Chen Zhu · Thodoris Lykouris · Jianfeng Chi · Mark Kozdoba · Arjun Nitin Bhagoji · Xiaoxia Wu · Jay Nandy · Michael T Smith · Bingyang Wen · Yuege Xie · Konstantinos Pitas · Suprosanna Shit · Maksym Andriushchenko · Dingli Yu · Gaël Letarte · Misha Khodak · Hussein Mozannar · Chara Podimata · James Foulds · Yizhen Wang · Huishuai Zhang · Ondrej Kuzelka · Alexander Levine · Nan Lu · Zakaria Mhammedi · Paul Viallard · Diana Cai · Lovedeep Gondara · James Lucas · Yasaman Mahdaviyeh · Aristide Baratin · Rishi Bommasani · Alessandro Barp · Andrew Ilyas · Kaiwen Wu · Jens Behrmann · Omar Rivasplata · Amir Nazemi · Aditi Raghunathan · Will Stephenson · Sahil Singla · Akhil Gupta · YooJung Choi · Yannic Kilcher · Clare Lyle · Edoardo Manino · Andrew Bennett · Zhi Xu · Niladri Chatterji · Emre Barut · Flavien Prost · Rodrigo Toro Icarte · Arno Blaas · Chulhee Yun · Sahin Lale · YiDing Jiang · Tharun Kumar Reddy Medini · Ashkan Rezaei · Alexander Meinke · Stephen Mell · Gary Kazantsev · Shivam Garg · Aradhana Sinha · Vishnu Lokhande · Geovani Rizk · Han Zhao · Aditya Kumar Akash · Jikai Hou · Ali Ghodsi · Matthias Hein · Tyler Sypherd · Yichen Yang · Anastasia Pentina · Pierre Gillot · Antoine Ledent · Guy Gur-Ari · Noah MacAulay · Tianzong Zhang -
2019 Poster: PAC-Bayes Un-Expected Bernstein Inequality »
Zakaria Mhammedi · Peter Grünwald · Benjamin Guedj -
2019 Poster: Dichotomize and Generalize: PAC-Bayesian Binary Activated Deep Neural Networks »
Gaël Letarte · Pascal Germain · Benjamin Guedj · Francois Laviolette -
2017 : Concluding remarks »
Francis Bach · Benjamin Guedj · Pascal Germain -
2017 : Overture »
Benjamin Guedj · Francis Bach · Pascal Germain -
2017 Workshop: (Almost) 50 shades of Bayesian Learning: PAC-Bayesian trends and insights »
Benjamin Guedj · Pascal Germain · Francis Bach -
2017 Poster: Joint distribution optimal transportation for domain adaptation »
Nicolas Courty · Rémi Flamary · Amaury Habrard · Alain Rakotomamonjy -
2016 Poster: Mapping Estimation for Discrete Optimal Transport »
Michaël Perrot · Nicolas Courty · Rémi Flamary · Amaury Habrard -
2016 Poster: beta-risk: a New Surrogate Risk for Learning from Weakly Labeled Data »
Valentina Zantedeschi · Rémi Emonet · Marc Sebban -
2015 Poster: Regressive Virtual Metric Learning »
Michaël Perrot · Amaury Habrard -
2014 Workshop: Second Workshop on Transfer and Multi-Task Learning: Theory meets Practice »
Urun Dogan · Tatiana Tommasi · Yoshua Bengio · Francesco Orabona · Marius Kloft · Andres Munoz · Gunnar Rätsch · Hal Daumé III · Mehryar Mohri · Xuezhi Wang · Daniel Hernández-lobato · Song Liu · Thomas Unterthiner · Pascal Germain · Vinay P Namboodiri · Michael Goetz · Christopher Berlind · Sigurd Spieckermann · Marta Soare · Yujia Li · Vitaly Kuznetsov · Wenzhao Lian · Daniele Calandriello · Emilie Morvant -
2014 Poster: Multilabel Structured Output Learning with Random Spanning Trees of Max-Margin Markov Networks »
Mario Marchand · Hongyu Su · Emilie Morvant · Juho Rousu · John Shawe-Taylor -
2009 Poster: From PAC-Bayes Bounds to KL Regularization »
Pascal Germain · Alexandre Lacasse · Francois Laviolette · Mario Marchand · Sara Shanian