Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning
Communication of model updates between client nodes and the central aggregating server is a major bottleneck in federated learning, especially in bandwidth-limited settings and high-dimensional models. Gradient quantization is an effective way of reducing the number of bits required to communicate each model update, albeit at the cost of having a higher error floor due to the higher variance of the stochastic gradients. In this work, we propose an adaptive quantization strategy called AdaQuantFL that aims to achieve communication efficiency as well as a low error floor by changing the number of quantization levels during the course of training. Experiments on training deep neural networks show that our method can converge in much fewer communicated bits as compared to fixed quantization level setups, with little or no impact on training and test accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningQuantizationSimilar Papers 제목 키워드 기반
FedDQ: Communication-Efficient Federated Learning with Descending Quantization
Federated learning (FL) is an emerging learning paradigm without violating users' privacy. However, large model size and frequent model aggregation cause serious communication bottleneck for FL. To reduce the communicati…
Federated LearningModel CompressionPrivacy PreservingQuantizationAccelerating Energy-Efficient Federated Learning in Cell-Free Networks with Adaptive Quantization
Federated Learning (FL) enables clients to share learning parameters instead of local data, reducing communication overhead. Traditional wireless networks face latency challenges with FL. In contrast, Cell-Free Massive M…
Federated LearningQuantizationCommunication-Efficient Adaptive Federated Learning
Federated learning is a machine learning training paradigm that enables clients to jointly train models without sharing their own localized data. However, the implementation of federated learning in practice still faces …
Federated LearningQuantizationStochastic OptimizationAQUILA: Communication Efficient Federated Learning with Adaptive Quantization in Device Selection Strategy
The widespread adoption of Federated Learning (FL), a privacy-preserving distributed learning methodology, has been impeded by the challenge of high communication overheads, typically arising from the transmission of lar…
Federated LearningPrivacy PreservingQuantizationEnhancing Communication Efficiency in FL with Adaptive Gradient Quantization and Communication Frequency Optimization
Federated Learning (FL) enables participant devices to collaboratively train deep learning models without sharing their data with the server or other devices, effectively addressing data privacy and computational concern…
Federated Learning