Deep Hierarchy Quantization Compression algorithm based on Dynamic Sampling
Unlike traditional distributed machine learning, federated learning stores data locally for training and then aggregates the models on the server, which solves the data security problem that may arise in traditional distributed machine learning. However, during the training process, the transmission of model parameters can impose a significant load on the network bandwidth. It has been pointed out that the vast majority of model parameters are redundant during model parameter transmission. In this paper, we explore the data distribution law of selected partial model parameters on this basis, and propose a deep hierarchical quantization compression algorithm, which further compresses the model and reduces the network load brought by data transmission through the hierarchical quantization of model parameters. And we adopt a dynamic sampling strategy for the selection of clients to accelerate the convergence of the model. Experimental results on different public datasets demonstrate the effectiveness of our algorithm.
Code (0)
등록된 구현이 없습니다.
Tasks
Federated LearningQuantizationSimilar Papers 제목 키워드 기반
Image Compression Based on Compressive Sensing: End-to-End Comparison with JPEG
We present an end-to-end image compression system based on compressive sensing. The presented system integrates the conventional scheme of compressive sampling and reconstruction with quantization and entropy coding. The…
Compressive SensingImage CompressionQuantizationTask-Based Graph Signal Compression
Graph signals arise in various applications, ranging from sensor networks to social media data. The high-dimensional nature of these signals implies that they often need to be compressed in order to be stored and transmi…
QuantizationDynamic Predictive Sampling Analog to Digital Converter for Sparse Signal Sensing
This paper presents a dynamic predictive sampling (DPS) based analog-to-digital converter (ADC) that provides a non-uniform sampling of input analog continuous-time signals. The processing unit generates a dynamic predic…
Data CompressionQuantizationDAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning
Federated Learning (FL) is a powerful technique for training a model on a server with data from several clients in a privacy-preserving manner. In FL, a server sends the model to every client, who then train the model lo…
Federated LearningPrivacy PreservingQuantizationTR-DQ: Time-Rotation Diffusion Quantization
Diffusion models have been widely adopted in image and video generation. However, their complex network architecture leads to high inference overhead for its generation process. Existing diffusion quantization methods pr…
Image GenerationQuantizationVideo Generation