Spatio-Temporal Communication Compression in Distributed Prime-Dual Flows
In this paper, we study distributed prime-dual flows for multi-agent optimization with spatio-temporal compressions. The central aim of multi-agent optimization is for a network of agents to collaboratively solve a system-level optimization problem with local objective functions and node-to-node communication by distributed algorithms. The scalability of such algorithms crucially depends on the complexity of the communication messages, and a number of communication compressors for distributed optimization have recently been proposed in the literature. First of all, we introduce a general spatio-temporal compressor characterized by the stability of the resulting dynamical system along the vector field of the compressor. We show that several important distributed optimization compressors such as the greedy sparsifier, the uniform quantizer, and the scalarizer all fall into the category of this spatio-temporal compressor. Next, we propose two distributed prime-dual flows with the spatio-temporal compressors being applied to local node states and local error states, respectively, and prove (exponential) convergence of the node trajectories to the global optimizer for (strongly) convex cost functions. Finally, a few numerical examples are present to illustrate our theoretical results.
Code (0)
등록된 구현이 없습니다.
Tasks
Distributed OptimizationSimilar Papers 제목 키워드 기반
Distributed Optimization by Network Flows with Spatio-Temporal Compression
Several data compressors have been proposed in distributed optimization frameworks of network systems to reduce communication overhead in large-scale applications. In this paper, we demonstrate that effective information…
Distributed OptimizationLinear Convergence of Distributed Compressed Optimization with Equality Constraints
In this paper, the distributed strongly convex optimization problem is studied with spatio-temporal compressed communication and equality constraints. For the case where each agent holds an distributed local equality con…
Temporal Predictive Coding for Gradient Compression in Distributed Learning
This paper proposes a prediction-based gradient compression method for distributed learning with event-triggered communication. Our goal is to reduce the amount of information transmitted from the distributed agents to t…
PredictionSparse Binary Compression: Towards Distributed Deep Learning with minimal Communication
Currently, progressively larger deep neural networks are trained on ever growing data corpora. As this trend is only going to increase in the future, distributed training schemes are becoming increasingly relevant. A maj…
BinarizationDeep LearningSpatiotemporal Entropy Model is All You Need for Learned Video Compression
The framework of dominant learned video compression methods is usually composed of motion prediction modules as well as motion vector and residual image compression modules, suffering from its complex structure and error…
AllImage Compressionmotion predictionMS-SSIM+2