Federated Learning with Intermediate Representation Regularization
In contrast to centralized model training that involves data collection, federated learning (FL) enables remote clients to collaboratively train a model without exposing their private data. However, model performance usually degrades in FL due to the heterogeneous data generated by clients of diverse characteristics. One promising strategy to maintain good performance is by limiting the local training from drifting far away from the global model. Previous studies accomplish this by regularizing the distance between the representations learned by the local and global models. However, they only consider representations from the early layers of a model or the layer preceding the output layer. In this study, we introduce FedIntR, which provides a more fine-grained regularization by integrating the representations of intermediate layers into the local training process. Specifically, FedIntR computes a regularization term that encourages the closeness between the intermediate layer representations of the local and global models. Additionally, FedIntR automatically determines the contribution of each layer's representation to the regularization term based on the similarity between local and global representations. We conduct extensive experiments on various datasets to show that FedIntR can achieve equivalent or higher performance compared to the state-of-the-art approaches. Our code is available at https://github.com/YLTun/FedIntR.
Code (1)
Tasks
Federated LearningSimilar Papers 제목 키워드 기반
FedDSR: Federated Deep Supervision and Regularization Towards Autonomous Driving
Federated Learning (FL) enables collaborative training of autonomous driving (AD) models across distributed vehicles while preserving data privacy. However, FL encounters critical challenges such as poor generalization a…
Semantic SegmentationFederated LearningAutonomous DrivingOmniISR: A Unified Framework for Centralized and Federated Learning via Intermediate Supervision and Regularization
The global deployment of edge intelligence operates across heterogeneous legal frameworks. While some regions permit centralized learning (CL) via cloud data aggregation, others enforce strict data localization, necessit…
Federated LearningFedDCL: a federated data collaboration learning as a hybrid-type privacy-preserving framework based on federated learning and data collaboration
Recently, federated learning has attracted much attention as a privacy-preserving integrated analysis that enables integrated analysis of data held by multiple institutions without sharing raw data. On the other hand, fe…
Federated LearningPrivacy PreservingMutual Information Regularization for Vertical Federated Learning
Vertical Federated Learning (VFL) is widely utilized in real-world applications to enable collaborative learning while protecting data privacy and safety. However, previous works show that parties without labels (passive…
Federated LearningVertical Federated LearningGuarding the Middle: Protecting Intermediate Representations in Federated Split Learning
Big data scenarios, where massive, heterogeneous datasets are distributed across clients, demand scalable, privacy-preserving learning methods. Federated learning (FL) enables decentralized training of machine learning (…
Federated Learning