paper-with-me

Papers

Distributed and Rate-Adaptive Feature Compression

2024-04-02 · Aditya Deshmukh, Venugopal V. Veeravalli, Gunjan Verma

We study the problem of distributed and rate-adaptive feature compression for linear regression. A set of distributed sensors collect disjoint features of regressor data. A fusion center is assumed to contain a pretrained linear regression model, trained on a dataset of the entire uncompressed data. At inference time, the sensors compress their observations and send them to the fusion center through communication-constrained channels, whose rates can change with time. Our goal is to design a feature compression {scheme} that can adapt to the varying communication constraints, while maximizing the inference performance at the fusion center. We first obtain the form of optimal quantizers assuming knowledge of underlying regressor data distribution. Under a practically reasonable approximation, we then propose a distributed compression scheme which works by quantizing a one-dimensional projection of the sensor data. We also propose a simple adaptive scheme for handling changes in communication constraints. We demonstrate the effectiveness of the distributed adaptive compression scheme through simulated experiments.

📄 PDF Abstract BibTeX arXiv:2404.02179

Code (0)

등록된 구현이 없습니다.

Tasks

Feature Compressionregression

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically
Linear Regression Linear Regression is a method for modelling a relationship between a dependent variable and independent variables. These models can be fit with numerous approaches. The most…

Similar Papers 제목 키워드 기반

Kimad: Adaptive Gradient Compression with Bandwidth Awareness

2023-12-13 · Jihao Xin, Ivan Ilin, Shunkang Zhang, Marco Canini 외

In distributed training, communication often emerges as a bottleneck. In response, we introduce Kimad, a solution that offers adaptive gradient compression. By consistently monitoring bandwidth, Kimad refines compression…

Communication-Compressed Adaptive Gradient Method for Distributed Nonconvex Optimization

2021-11-01 · Yujia Wang, Lu Lin, Jinghui Chen

Due to the explosion in the size of the training datasets, distributed learning has received growing interest in recent years. One of the major bottlenecks is the large communication cost between the central server and t…

Compressed Communication for Distributed Training: Adaptive Methods and System

2021-05-17 · Yuchen Zhong, Cong Xie, Shuai Zheng, Haibin Lin

Communication overhead severely hinders the scalability of distributed machine learning systems. Recently, there has been a growing interest in using gradient compression to reduce the communication overhead of the distr…

On Distributed Adaptive Optimization with Gradient Compression

2022-05-11 · ICLR 2022 4 · Xiaoyun Li, Belhal Karimi, Ping Li

We study COMP-AMS, a distributed optimization framework based on gradient averaging and adaptive AMSGrad algorithm. Gradient compression with error feedback is applied to reduce the communication cost in the gradient tra…

Distributed Optimization

D-DPCC: Deep Dynamic Point Cloud Compression via 3D Motion Prediction

2022-05-02 · Tingyu Fan, Linyao Gao, Yiling Xu, Zhu Li 외

The non-uniformly distributed nature of the 3D dynamic point cloud (DPC) brings significant challenges to its high-efficient inter-frame compression. This paper proposes a novel 3D sparse convolution-based Deep Dynamic P…

Motion CompensationMotion Estimationmotion predictionOptical Flow Estimation