Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM
In this paper, we propose a communication-efficiently decentralized machine learning framework that solves a consensus optimization problem defined over a network of inter-connected workers. The proposed algorithm, Censored and Quantized Generalized GADMM (CQ-GGADMM), leverages the worker grouping and decentralized learning ideas of Group Alternating Direction Method of Multipliers (GADMM), and pushes the frontier in communication efficiency by extending its applicability to generalized network topologies, while incorporating link censoring for negligible updates after quantization. We theoretically prove that CQ-GGADMM achieves the linear convergence rate when the local objective functions are strongly convex under some mild assumptions. Numerical simulations corroborate that CQ-GGADMM exhibits higher communication efficiency in terms of the number of communication rounds and transmit energy consumption without compromising the accuracy and convergence speed, compared to the censored decentralized ADMM, and the worker grouping method of GADMM.
Code (0)
등록된 구현이 없습니다.
Tasks
QuantizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Distributed Finite Time k-means Clustering with Quantized Communucation and Transmission Stopping
In this paper, we present a distributed algorithm which implements the $k$-means algorithm in a distributed fashion for multi-agent systems with directed communication links. The goal of $k$-means is to partition the net…
ClusteringQC-ODKLA: Quantized and Communication-Censored Online Decentralized Kernel Learning via Linearized ADMM
This paper focuses on online kernel learning over a decentralized network. Each agent in the network receives continuous streaming data locally and works collaboratively to learn a nonlinear prediction function that is g…
QuantizationCommunication-Censored Distributed Stochastic Gradient Descent
This paper develops a communication-efficient algorithm to solve the stochastic optimization problem defined over a distributed network, aiming at reducing the burdensome communication in applications such as distributed…
QuantizationStochastic OptimizationDistributed Estimation with Quantized Measurements and Communication over Markovian Switching Topologies
This paper addresses distributed parameter estimation in stochastic dynamic systems with quantized measurements, constrained by quantized communication and Markovian switching directed topologies. To enable accurate reco…
parameter estimationDistributed Computation of Exact Average Degree and Network Size in Finite Number of Steps under Quantized Communication
We consider the problems of computing the average degree and the size of a given network in a distributed fashion under quantized communication. We present two distributed algorithms which rely on quantized operation (i.…
Quantization