Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learning
Federated learning (FL) is a novel machine learning setting that enables on-device intelligence via decentralized training and federated optimization. Deep neural networks' rapid development facilitates the learning techniques for modeling complex problems and emerges into federated deep learning under the federated setting. However, the tremendous amount of model parameters burdens the communication network with a high load of transportation. This paper introduces two approaches for improving communication efficiency by dynamic sampling and top-$k$ selective masking. The former controls the fraction of selected client models dynamically, while the latter selects parameters with top-$k$ largest values of difference for federated updating. Experiments on convolutional image classification and recurrent language modeling are conducted on three public datasets to show our proposed methods' effectiveness.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated Learningimage-classificationImage ClassificationLanguage ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Selective Attention Federated Learning: Improving Privacy and Efficiency for Clinical Text Classification
Federated Learning (FL) faces major challenges regarding communication overhead and model privacy when training large language models (LLMs), especially in healthcare applications. To address these, we introduce Selectiv…
Clinical Concept ExtractionFederated Learningtext-classificationText ClassificationGlueFL: Reconciling Client Sampling and Model Masking for Bandwidth Efficient Federated Learning
Federated learning (FL) is an effective technique to directly involve edge devices in machine learning training while preserving client privacy. However, the substantial communication overhead of FL makes training challe…
Federated LearningModel CompressionPrivacy-Preserving Federated Action Recognition via Differentially Private Selective Tuning and Efficient Communication
Federated video action recognition enables collaborative model training without sharing raw video data, yet remains vulnerable to two key challenges: \textit{model exposure} and \textit{communication overhead}. Gradients…
Action RecognitionFederated Graph Learning with Adaptive Importance-based Sampling
For privacy-preserving graph learning tasks involving distributed graph datasets, federated learning (FL)-based GCN (FedGCN) training is required. A key challenge for FedGCN is scaling to large-scale graphs, which typica…
Federated LearningGraph LearningGraph SamplingPrivacy PreservingEnergizing Federated Learning via Filter-Aware Attention
Federated learning (FL) is a promising distributed paradigm, eliminating the need for data sharing but facing challenges from data heterogeneity. Personalized parameter generation through a hypernetwork proves effective,…
Federated Learning