Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning
Federated Learning is an evolving machine learning paradigm, in which multiple clients perform computations based on their individual private data, interspersed by communication with a remote server. A common strategy to curtail communication costs is Local Training, which consists in performing multiple local stochastic gradient descent steps between successive communication rounds. However, the conventional approach to local training overlooks the practical necessity for client-specific personalization, a technique to tailor local models to individual needs. We introduce Scafflix, a novel algorithm that efficiently integrates explicit personalization with local training. This innovative approach benefits from these two techniques, thereby achieving doubly accelerated communication, as we demonstrate both in theory and practice.
Code (1)
Tasks
Federated LearningSimilar Papers 제목 키워드 기반
Federated Learning of a Mixture of Global and Local Models
We propose a new optimization formulation for training federated learning models. The standard formulation has the form of an empirical risk minimization problem constructed to find a single global model trained from the…
Federated LearningStrategies for Improving Communication Efficiency in Distributed and Federated Learning: Compression, Local Training, and Personalization
Distributed and federated learning are essential paradigms for training models across decentralized data sources while preserving privacy, yet communication overhead remains a major bottleneck. This dissertation explores…
Federated LearningModel CompressionDistributed Differential Graphical Game for Control of Double-Integrator Multi-Agent Systems with Input Delay
This paper studies cooperative control of noncooperative double-integrator multi-agent systems (MASs) with input delay on connected directed graphs in the context of a differential graphical game (DGG). In the distribute…
NEXUS-MI: Communication-Aware Federated Personalization for Gateway-Coordinated Motor-Imagery Brain-Computer Interfaces
Electroencephalography (EEG)-based motor-imagery brain-computer interfaces (MI-BCIs) vary across subjects and sessions, complicating personalization from limited calibration data. Federated learning can exploit shared re…
Federated LearningPearl: Personalizing Large Language Model Writing Assistants with Generation-Calibrated Retrievers
Powerful large language models have facilitated the development of writing assistants that promise to significantly improve the quality and efficiency of composition and communication. However, a barrier to effective ass…
Language ModelingLanguage ModellingLarge Language ModelRetrieval