Timezone: »

 
FLoRA: Single-shot Hyper-parameter Optimization for Federated Learning
Yi Zhou · Parikshit Ram · Theodoros Salonidis · Nathalie Baracaldo · Horst Samulowitz · Heiko Ludwig

@
in New Frontiers in Federated Learning: Privacy, Fairness, Robustness, Personalization and Data Ownership »

We address the relatively unexplored problem of hyper-parameter optimization (HPO) for federated learning (FL-HPO). We introduce {\bf F}ederated {\bf Lo}ss Su{\bf R}face {\bf A}ggregation (FLoRA), the first FL-HPO solution framework that can address use cases of tabular data and gradient boosting training algorithms in addition to stochastic gradient descent/neural networks commonly addressed in the FL literature. The framework enables single-shot FL-HPO, by first identifying a good set of hyper-parameters that are used in a {\em single} FL training. Thus, it enables FL-HPO solutions with minimal additional communication overhead compared to FL training without HPO. Our empirical evaluation of FLoRA for Gradient Boosted Decision Trees on seven OpenML data sets demonstrates significant model accuracy improvements over the considered baseline, and robustness to increasing number of parties involved in FL-HPO training.

Author Information

Yi Zhou (IBM Almaden Research Center)
Parikshit Ram (IBM Research AI)
Theodoros Salonidis (IBM Research)
Nathalie Baracaldo (IBM Research AI)

Nathalie Baracaldo leads the AI Security and Privacy Solutions team and is a Research Staff Member at IBM’s Almaden Research Center in San Jose, CA. Nathalie is passionate about delivering machine learning solutions that are highly accurate, withstand adversarial attacks and protect data privacy. Her team focuses on two main areas: federated learning, where models are trained without directly accessing training data and adversarial machine learning, where defenses are designed to withstand potential attacks to the machine learning pipeline. Nathalie is the primary investigator for the DARPA program Guaranteeing AI Robustness Against Deception (GARD), where AI security is investigated. Her team contributes to the Adversarial Robustness 360 Toolbox (ART). Nathalie is also the co-editor of the book: “Federated Learning: A Comprehensive Overview of Methods and Applications”, 2022 available in paper and as e-book in Springer, Apple books and Amazon. Nathalie's primary research interests lie at the intersection of information security, privacy and trust. As part of her work, she has also designed and implemented secure systems in the areas of cloud computing, Platform as a Service, secure data sharing and Internet of the Things. She has also contributed to projects to design scalable systems that monitor, manage performance and manage service level agreements in cloud environments. In 2020, Nathalie received the IBM Master Inventor distinction for her contributions to the IBM Intellectual Property and innovation. Nathalie also received the 2021 Corporate Technical Recognition, one of the highest recognitions provided to IBMers for breakthrough technical achievements that have led to notable market and industry success for IBM. This recognition was awarded for Nathalie's contribution to the Trusted AI initiative. Nathalie is associated Editor IEEE Transactions on Service Computing. Nathalie received her Ph.D. degree from the University of Pittsburgh in 2016. Her dissertation focused on preventing insider threats through the use of adaptive access control systems that integrate multiple sources of contextual information. Some of the topics that she has explored in the past include secure storage systems, privacy in online social networks, secure interoperability in distributed systems, risk management and trust evaluation. During her Ph.D. studies she received the 2014 Allen Kent Award for Outstanding Contributions to the Graduate Program in Information Science by the School of Information Sciences at the University of Pittsburgh. Nathalie also holds a master’s degree with Cum Laude distinction in computer sciences from the Universidad de los Andes, Colombia. Prior to that, she earned two undergraduate degrees in Computer Science and Industrial Engineering at the same university.

Horst Samulowitz (IBM Research)
Heiko Ludwig (IBM Research AI)

More from the Same Authors