Device-Wise Federated Network Pruning
Neural network pruning particularly channel pruning is a widely used technique for compressing deep learning models to enable their deployment on edge devices with limited resources. Typically redundant weights or structures are removed to achieve the target resource budget. Although data-driven pruning approaches have proven to be more effective they cannot be directly applied to federated learning (FL) which has emerged as a popular technique in edge computing applications because of distributed and confidential datasets. In response to this challenge we design a new network pruning method for FL. We propose device-wise sub-networks for each device assuming that the data distribution is similar within each device. These sub-networks are generated through sub-network embeddings and a hypernetwork. To further minimize memory usage and communication costs we permanently prune the full model to remove weights that are not useful for all devices. During the FL process we simultaneously train the device-wise sub-networks and the base sub-network to facilitate the pruning process. We then finetune the pruned model with device-wise sub-networks to regain performance. Moreover we provided the theoretical guarantee of convergence for our method. Our method achieves better performance and resource trade-off than other well-established network pruning baselines as demonstrated through extensive experiments on CIFAR-10 CIFAR-100 and TinyImageNet.
Code (1)
Tasks
Edge-computingFederated LearningNetwork PruningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FedLP: Layer-wise Pruning Mechanism for Communication-Computation Efficient Federated Learning
Federated learning (FL) has prevailed as an efficient and privacy-preserved scheme for distributed learning. In this work, we mainly focus on the optimization of computation and communication in FL from a view of pruning…
Federated LearningFedOBP: Federated Optimal Brain Personalization through Cloud-Edge Element-wise Decoupling
Federated Learning (FL) faces challenges from client data heterogeneity and resource-constrained mobile devices, which can degrade model accuracy. Personalized Federated Learning (PFL) addresses this issue by adapting sh…
Personalized Federated LearningCA-HFP: Curvature-Aware Heterogeneous Federated Pruning with Model Reconstruction
Federated learning on heterogeneous edge devices requires personalized compression while preserving aggregation compatibility and stable convergence. We present Curvature-Aware Heterogeneous Federated Pruning (CA-HFP), a…
Federated LearningFederated Pruning: Improving Neural Network Efficiency with Federated Learning
Automatic Speech Recognition models require large amount of speech data for training, and the collection of such data often leads to privacy concerns. Federated learning has been widely used and is considered to be an ef…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Federated Learningspeech-recognition+1Automatic Structured Pruning for Efficient Architecture in Federated Learning
In Federated Learning (FL), training is conducted on client devices, typically with limited computational resources and storage capacity. To address these constraints, we propose an automatic pruning scheme tailored for …
Federated Learning