From Aggregation to Guidance: Strategies for Personalized Federated Fine-Tuning of Foundation Models

Federated learning (FL) enables collaborative fine-tuning of pretrained foundation models in privacy-sensitive settings without directly sharing raw data. Personalized federated learning (PFL) further addresses client-side heterogeneity by learning models tailored to each client's local tasks while still benefiting from cross-client collaboration. In this work, we study strategies to improve the effectiveness of personalized federated fine-tuning of Large Language Models (LLMs) using LoRA. On the server side, we explore advanced aggregation strategies that go beyond simple parameter averaging, drawing on model merging techniques to construct a robust global model.On the client side, we investigate different ways to leverage the global model to guide local learning, including hard initialization, parameter regularization, and function-space guidance via knowledge distillation, and propose a similarity-based adjustment strategy to further improve local learning. Empirical results on the Super NaturalInstructions dataset demonstrate that careful design of both server- and client-side strategies has the potential to improve PFL performance, providing insights for developing more effective PFL learning frameworks for fine-tuning LLMs for heterogeneous client tasks.