Timezone: »

Federated Functional Variational Inference
Michael Hutchinson · Matthias Reisser · Christos Louizos
Event URL: https://openreview.net/forum?id=gJPaFj1m0_X »

Traditional federated learning (FL) involves optimizing point estimates for the parameters of the server model via a maximum likelihood objective. While models trained with such objectives show competitive predictive accuracy, they are poorly calibrated and provide no reliable uncertainty estimates. Well calibrated uncertainty is, however, important in safety critical applications of FL such as self-driving cars and healthcare. In this work we propose several methods to train Bayesian neural networks, networks providing uncertainty over their model parameters, in FL. We introduce baseline methods that employ priors in and do inference on the weight-space of the network. We also propose two function-space inference methods. These build upon recent work in functional variational inference to posit prior distributions in and do inference on the function-space of the network. These two approaches are based on Federated Averaging (FedAvg) and Expectation-Maximization (EM). We compare these function-space methods to their weight-space counterparts.

Author Information

Michael Hutchinson (University of Oxford)

Hi I'm Michael, a first year DPhil student at Oxford under the supervision of Yee Whye Teh and Max Welling. I'm interested in Probabalistic Machine Leanring in general, with a specific interests in distributed learning, generative modelling and uncertianty at a functional level.

Matthias Reisser (Qualcomm AI Research)
Christos Louizos (University of Amsterdam)

More from the Same Authors

  • 2022 : Decentralized Learning with Random Walks and Communication-Efficient Adaptive Optimization »
    Aleksei Triastcyn · Matthias Reisser · Christos Louizos
  • 2022 : Spectral Diffusion Processes »
    Angus Phillips · Thomas Seror · Michael Hutchinson · Valentin De Bortoli · Arnaud Doucet · Emile Mathieu
  • 2022 Poster: Riemannian Score-Based Generative Modelling »
    Valentin De Bortoli · Emile Mathieu · Michael Hutchinson · James Thornton · Yee Whye Teh · Arnaud Doucet
  • 2021 Poster: Vector-valued Gaussian Processes on Riemannian Manifolds via Gauge Independent Projected Kernels »
    Michael Hutchinson · Alexander Terenin · Viacheslav Borovitskiy · So Takao · Yee Teh · Marc Deisenroth
  • 2019 : Poster Session »
    Clement Canonne · Kwang-Sung Jun · Seth Neel · Di Wang · Giuseppe Vietri · Liwei Song · Jonathan Lebensold · Huanyu Zhang · Lovedeep Gondara · Ang Li · FatemehSadat Mireshghallah · Jinshuo Dong · Anand D Sarwate · Antti Koskela · Joonas Jälkö · Matt Kusner · Dingfan Chen · Mi Jung Park · Ashwin Machanavajjhala · Jayashree Kalpathy-Cramer · · Vitaly Feldman · Andrew Tomkins · Hai Phan · Hossein Esfandiari · Mimansa Jaiswal · Mrinank Sharma · Jeff Druce · Casey Meehan · Zhengli Zhao · Hsiang Hsu · Davis Railsback · Abraham Flaxman · · Julius Adebayo · Aleksandra Korolova · Jiaming Xu · Naoise Holohan · Samyadeep Basu · Matthew Joseph · My Thai · Xiaoqian Yang · Ellen Vitercik · Michael Hutchinson · Chenghong Wang · Gregory Yauney · Yuchao Tao · Chao Jin · Si Kai Lee · Audra McMillan · Rauf Izmailov · Jiayi Guo · Siddharth Swaroop · Tribhuvanesh Orekondy · Hadi Esmaeilzadeh · Kevin Procopio · Alkis Polyzotis · Jafar Mohammadi · Nitin Agrawal