Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning

2020-11-08 · Nader Bouacida, Jiahui Hou, Hui Zang, Xin Liu

With more regulations tackling users' privacy-sensitive data protection in recent years, access to such data has become increasingly restricted and controversial. To exploit the wealth of data generated and located at distributed entities such as mobile phones, a revolutionary decentralized machine learning setting, known as Federated Learning, enables multiple clients located at different geographical locations to collaboratively learn a machine learning model while keeping all their data on-device. However, the scale and decentralization of federated learning present new challenges. Communication between the clients and the server is considered a main bottleneck in the convergence time of federated learning. In this paper, we propose and study Adaptive Federated Dropout (AFD), a novel technique to reduce the communication costs associated with federated learning. It optimizes both server-client communications and computation costs by allowing clients to train locally on a selected subset of the global model. We empirically show that this strategy, combined with existing compression methods, collectively provides up to 57x reduction in convergence time. It also outperforms the state-of-the-art solutions for communication efficiency. Furthermore, it improves model generalization by up to 1.7%.

📄 PDF Abstract BibTeX arXiv:2011.04050

Code (0)

등록된 구현이 없습니다.

Tasks

BIG-bench Machine Learning Federated Learning

Methods 이 논문이 사용한 방법론

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

FedBIAD: Communication-Efficient and Accuracy-Guaranteed Federated Learning with Bayesian Inference-Based Adaptive Dropout

2023-07-14 · Jingjing Xue, Min Liu, Sheng Sun, Yuwei Wang 외

Federated Learning (FL) emerges as a distributed machine learning paradigm without end-user data transmission, effectively avoiding privacy leakage. Participating devices in FL are usually bandwidth-constrained, and the …

Bayesian Inference Federated Learning image-classification Image Classification

FedDD: Toward Communication-efficient Federated Learning with Differential Parameter Dropout

2023-08-31 · Zhiying Feng, Xu Chen, Qiong Wu, Wen Wu 외

Federated Learning (FL) requires frequent exchange of model parameters, which leads to long communication delay, especially when the network environments of clients vary greatly. Moreover, the parameter server needs to w…

Federated Learning

SPD-CFL: Stepwise Parameter Dropout for Efficient Continual Federated Learning

2024-05-15 · Yuning Yang, Han Yu, Chuan Sun, Tianrun Gao 외

Federated Learning (FL) is a collaborative machine learning paradigm for training models on local sensitive data with privacy protection. Pre-trained transformer-based models have emerged as useful foundation models (FMs…

Continual Learning Federated Learning parameter-efficient fine-tuning Transfer Learning

Efficient Federated Learning with Heterogeneous Data and Adaptive Dropout

2025-07-14 · Ji Liu, Beichen Ma, Qiaolin Yu, Ruoming Jin 외

Federated Learning (FL) is a promising distributed machine learning approach that enables collaborative training of a global model using multiple edge devices. The data distributed among the edge devices is highly hetero…