Correlated quantization for distributed mean estimation and optimization
We study the problem of distributed mean estimation and optimization under communication constraints. We propose a correlated quantization protocol whose leading term in the error guarantee depends on the mean deviation of data points rather than only their absolute range. The design doesn't need any prior knowledge on the concentration property of the dataset, which is required to get such dependence in previous works. We show that applying the proposed protocol as sub-routine in distributed optimization algorithms leads to better convergence rates. We also prove the optimality of our protocol under mild assumptions. Experimental results show that our proposed algorithm outperforms existing mean estimation protocols on a diverse set of tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
Distributed OptimizationQuantizationSimilar Papers 제목 키워드 기반
Sensor Selection and Distributed Quantization for Energy Efficiency in Massive MTC
This paper presents an estimation approach within the framework of uplink massive machine-type communications (mMTC) that considers the energy limitations of the devices. We focus on a scenario where a group of sensors o…
QuantizationCorrelated Quantization for Faster Nonconvex Distributed Optimization
Quantization (Alistarh et al., 2017) is an important (stochastic) compression technique that reduces the volume of transmitted bits during each communication round in distributed model training. Suresh et al. (2022) intr…
Distributed OptimizationQuantizationDistributed Mean Estimation with Limited Communication
Motivated by the need for distributed learning and optimization algorithms with low communication cost, we study communication efficient algorithms for distributed mean estimation. Unlike previous works, we make no proba…
QuantizationWyner-Ziv Estimators for Distributed Mean Estimation with Side Information and Optimization
Communication efficient distributed mean estimation is an important primitive that arises in many distributed learning and optimization scenarios such as federated learning. Without any probabilistic assumptions on the u…
Distributed OptimizationFederated LearningCorrelated Privacy Mechanisms for Differentially Private Distributed Mean Estimation
Differentially private distributed mean estimation (DP-DME) is a fundamental building block in privacy-preserving federated learning, where a central server estimates the mean of $d$-dimensional vectors held by $n$ users…
Federated LearningPrivacy Preserving