paper-with-me

Papers

Distributed Inference with Sparse and Quantized Communication

2020-04-02 · Aritra Mitra, John A. Richards, Saurabh Bagchi, Shreyas Sundaram

We consider the problem of distributed inference where agents in a network observe a stream of private signals generated by an unknown state, and aim to uniquely identify this state from a finite set of hypotheses. We focus on scenarios where communication between agents is costly, and takes place over channels with finite bandwidth. To reduce the frequency of communication, we develop a novel event-triggered distributed learning rule that is based on the principle of diffusing low beliefs on each false hypothesis. Building on this principle, we design a trigger condition under which an agent broadcasts only those components of its belief vector that have adequate innovation, to only those neighbors that require such information. We prove that our rule guarantees convergence to the true state exponentially fast almost surely despite sparse communication, and that it has the potential to significantly reduce information flow from uninformative agents to informative agents. Next, to deal with finite-precision communication channels, we propose a distributed learning rule that leverages the idea of adaptive quantization. We show that by sequentially refining the range of the quantizers, every agent can learn the truth exponentially fast almost surely, while using just $1$ bit to encode its belief on each hypothesis. For both our proposed algorithms, we rigorously characterize the trade-offs between communication-efficiency and the learning rate.

📄 PDF Abstract BibTeX arXiv:2004.01302

Code (0)

등록된 구현이 없습니다.

Tasks

Quantization

Similar Papers 제목 키워드 기반

A Linear Speedup Analysis of Distributed Deep Learning with Sparse and Quantized Communication

2018-12-01 · NeurIPS 2018 12 · Peng Jiang, Gagan Agrawal

The large communication overhead has imposed a bottleneck on the performance of distributed Stochastic Gradient Descent (SGD) for training deep neural networks. Previous works have demonstrated the potential of using gr…

Quantization

Quantized Adaptive Subgradient Algorithms and Their Applications

2022-08-11 · Ke Xu, Jianqiao Wangni, Yifan Zhang, Deheng Ye 외

Data explosion and an increase in model size drive the remarkable advances in large-scale machine learning, but also make model training time-consuming and model storage difficult. To address the above issues in the dist…

Quantization

Distributed Estimation with Quantized Measurements and Communication over Markovian Switching Topologies

2024-12-14 · Ying Wang, Jian Guo, Yanlong Zhao, Ji-Feng Zhang

This paper addresses distributed parameter estimation in stochastic dynamic systems with quantized measurements, constrained by quantized communication and Markovian switching directed topologies. To enable accurate reco…

parameter estimation

Distributed Computation of Exact Average Degree and Network Size in Finite Number of Steps under Quantized Communication

2022-11-29 · Apostolos I. Rikos, Themistoklis Charalambous, Christoforos N. Hadjicostis, Karl H. Johansson

We consider the problems of computing the average degree and the size of a given network in a distributed fashion under quantized communication. We present two distributed algorithms which rely on quantized operation (i.…

Quantization

Distributed Average Consensus under Quantized Communication via Event-Triggered Mass Splitting

2019-07-23

We study the distributed average consensus problem in multi-agent systems with directed communication links that are subject to quantized information flow. The goal of distributed average consensus is for the nodes, each…

Quantization