Adaptive Parameterization of Deep Learning Models for Federated Learning
Federated Learning offers a way to train deep neural networks in a distributed fashion. While this addresses limitations related to distributed data, it incurs a communication overhead as the model parameters or gradients need to be exchanged regularly during training. This can be an issue with large scale distribution of learning tasks and negate the benefit of the respective resource distribution. In this paper, we we propose to utilise parallel Adapters for Federated Learning. Using various datasets, we show that Adapters can be incorporated to different Federated Learning techniques. We highlight that our approach can achieve similar inference performance compared to training the full model while reducing the communication overhead by roughly 90%. We further explore the applicability of Adapters in cross-silo and cross-device settings, as well as different non-IID data distributions.
Code (0)
등록된 구현이 없습니다.
Tasks
Deep LearningFederated LearningSimilar Papers 제목 키워드 기반
Differential Privacy Personalized Federated Learning Based on Dynamically Sparsified Client Updates
Personalized federated learning is extensively utilized in scenarios characterized by data heterogeneity, facilitating more efficient and automated local training on data-owning terminals. This includes the automated sel…
Federated LearningPersonalized Federated LearningExpressive variational quantum circuits provide inherent privacy in federated learning
Federated learning has emerged as a viable distributed solution to train machine learning models without the actual need to share data with the central aggregator. However, standard neural network-based federated learnin…
Federated LearningQuantum Machine LearningFedSVD: Adaptive Orthogonalization for Private Federated Learning with LoRA
Low-Rank Adaptation (LoRA), which introduces a product of two trainable low-rank matrices into frozen pre-trained weights, is widely used for efficient fine-tuning of language models in federated learning (FL). However, …
Federated LearningNoise-Resilient Quantum Aggregation on NISQ for Federated ADAS Learning
Advanced Driver Assistance Systems (ADAS) increasingly employ Federated Learning (FL) to collaboratively train models across distributed vehicular nodes while preserving data privacy. Yet, conventional FL aggregation rem…
Federated LearningFedHQL: Federated Heterogeneous Q-Learning
Federated Reinforcement Learning (FedRL) encourages distributed agents to learn collectively from each other's experience to improve their performance without exchanging their raw trajectories. The existing work on FedRL…
Q-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)