Adaptive Channel Sparsity for Federated Learning Under System Heterogeneity
Owing to the non-i.i.d. nature of client data, channel neurons in federated-learned models may specialize to distinct features for different clients. Yet, existing channel-sparse federated learning (FL) algorithms prescribe fixed sparsity strategies for client models, and may thus prevent clients from training channel neurons collaboratively. To minimize the impact of sparsity on FL convergence, we propose Flado to improve the alignment of client model update trajectories by tailoring the sparsities of individual neurons in each client. Empirical results show that while other sparse methods are surprisingly impactful to convergence, Flado can not only attain the highest task accuracies with unlimited budget across a range of datasets, but also significantly reduce the amount of FLOPs required for training more than by 10x under the same communications budget, and push the Pareto frontier of communication/computation trade-off notably further than competing FL algorithms.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningSimilar Papers 제목 키워드 기반
Channel-Adaptive Robust Aggregation for Over-the-Air Federated Learning in Heterogeneous Networks
The growing demand for privacy-preserving, data-intensive applications such as IoT, augmented reality, and autonomous systems positions Federated Learning (FL) as a key enabler in 6G networks. Over-the-Air FL (OTA-FL) le…
Federated LearningCompressive Sensing Based Adaptive Active User Detection and Channel Estimation: Massive Access Meets Massive MIMO
This paper considers massive access in massive multiple-input multiple-output (MIMO) systems and proposes an adaptive active user detection and channel estimation scheme based on compressive sensing. By exploiting the sp…
Compressive SensingFederated learning over physical channels: adaptive algorithms with near-optimal guarantees
In federated learning, communication cost can be significantly reduced by transmitting the information over the air through physical channels. In this paper, we propose a new class of adaptive federated stochastic gradie…
Federated LearningPeLAP-A: Adaptive Latent Pruning for Lightweight Latent Diffusion Models
Latent diffusion models achieve strong generative performance by operating in a compressed latent space produced by a variational autoencoder (VAE). However, it remains unclear whether all latent channels contribute equa…
Energy-Efficient Channel Decoding for Wireless Federated Learning: Convergence Analysis and Adaptive Design
One of the most critical challenges for deploying distributed learning solutions, such as federated learning (FL), in wireless networks is the limited battery capacity of mobile clients. While it is a common belief that …
Federated Learning