Federated Learning Based on Dynamic Regularization
We propose a novel federated learning method for distributively training neural network models, where the server orchestrates cooperation between a subset of randomly chosen devices in each round. We view Federated Learning problem primarily from a communication perspective and allow more device level computations to save transmission costs. We point out a fundamental dilemma, in that the minima of the local-device level empirical loss are inconsistent with those of the global empirical loss. Different from recent prior works, that either attempt inexact minimization or utilize devices for parallelizing gradient computation, we propose a dynamic regularizer for each device at each round, so that in the limit the global and device solutions are aligned. We demonstrate both through empirical results on real and synthetic data as well as analytical results that our scheme leads to efficient training, in both convex and non-convex settings, while being fully agnostic to device heterogeneity and robust to large number of devices, partial participation and unbalanced data.
Code (5)
Tasks
Federated LearningSimilar Papers 제목 키워드 기반
Dynamic Heterogeneous Federated Learning with Multi-Level Prototypes
Federated learning shows promise as a privacy-preserving collaborative learning technique. Existing heterogeneous federated learning mainly focuses on skewing the label distribution across clients. However, most approach…
Federated LearningPrivacy PreservingGradient Inversion in Federated Reinforcement Learning
Federated reinforcement learning (FRL) enables distributed learning of optimal policies while preserving local data privacy through gradient sharing.However, FRL faces the risk of data privacy leaks, where attackers expl…
Reinforcement LearningFederated LearningAutonomous DrivingFederated Learning with Projected Trajectory Regularization
Federated learning enables joint training of machine learning models from distributed clients without sharing their local data. One key challenge in federated learning is to handle non-identically distributed data across…
Federated LearningHierarchical Federated Learning with Dynamic Clustering and Adaptive Regularization for Robust Infrastructure Inspection
The deployment of data-driven computer vision models for structural health monitoring (SHM) is heavily constrained by the data silo dilemma due to stringent privacy and security regulations. While federated learning (FL)…
Federated LearningUniVarFL: Uniformity and Variance Regularized Federated Learning for Heterogeneous Data
Federated Learning (FL) often suffers from severe performance degradation when faced with non-IID data, largely due to local classifier bias. Traditional remedies such as global model regularization or layer freezing eit…
Federated Learning