Statistical Estimation and Inference via Local SGD in Federated Learning
Federated Learning (FL) makes a large amount of edge computing devices (e.g., mobile phones) jointly learn a global model without data sharing. In FL, data are generated in a decentralized manner with high heterogeneity. This paper studies how to perform statistical estimation and inference in the federated setting. We analyze the so-called Local SGD, a multi-round estimation procedure that uses intermittent communication to improve communication efficiency. We first establish a {\it functional central limit theorem} that shows the averaged iterates of Local SGD weakly converge to a rescaled Brownian motion. We next provide two iterative inference methods: the {\it plug-in} and the {\it random scaling}. Random scaling constructs an asymptotically pivotal statistic for inference by using the information along the whole Local SGD path. Both the methods are communication efficient and applicable to online data. Our theoretical and empirical results show that Local SGD simultaneously achieves both statistical efficiency and communication efficiency.
Code (0)
등록된 구현이 없습니다.
Tasks
Edge-computingFederated LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Differentially Private Federated Learning: Servers Trustworthiness, Estimation, and Statistical Inference
Differentially private federated learning is crucial for maintaining privacy in distributed environments. This paper investigates the challenges of high-dimensional estimation and inference under the constraints of diffe…
Federated LearningFederated Averaging Langevin Dynamics: Toward a unified theory and new algorithms
This paper focuses on Bayesian inference in a federated learning context (FL). While several distributed MCMC algorithms have been proposed, few consider the specific limitations of FL such as communication bottlenecks a…
Bayesian InferenceFederated LearningFederated PCA and Estimation for Spiked Covariance Matrices: Optimal Rates and Efficient Algorithm
Federated Learning (FL) has gained significant recent attention in machine learning for its enhanced privacy and data security, making it indispensable in fields such as healthcare, finance, and personalized services. Th…
Federated LearningAdaptive Personalized Federated Learning via Multi-task Averaging of Kernel Mean Embeddings
Personalized Federated Learning (PFL) enables a collection of agents to collaboratively learn individual models without sharing raw data. We propose a new PFL approach in which each agent optimizes a weighted combination…
Personalized Federated LearningFederated Prediction-Powered Inference from Decentralized Data
In various domains, the increasing application of machine learning allows researchers to access inexpensive predictive data, which can be utilized as auxiliary data for statistical inference. Although such data are often…
Federated LearningPredictionvalid