Lightweight Compression of Intermediate Neural Network Features for Collaborative Intelligence
In collaborative intelligence applications, part of a deep neural network (DNN) is deployed on a lightweight device such as a mobile phone or edge device, and the remaining portion of the DNN is processed where more computing resources are available, such as in the cloud. This paper presents a novel lightweight compression technique designed specifically to quantize and compress the features output by the intermediate layer of a split DNN, without requiring any retraining of the network weights. Mathematical models for estimating the clipping and quantization error of ReLU and leaky-ReLU activations at this intermediate layer are developed and used to compute optimal clipping ranges for coarse quantization. We also present a modified entropy-constrained design algorithm for quantizing clipped activations. When applied to popular object-detection and classification DNNs, we were able to compress the 32-bit floating point intermediate activations down to 0.6 to 0.8 bits, while keeping the loss in accuracy to less than 1%. When compared to HEVC, we found that the lightweight codec consistently provided better inference accuracy, by up to 1.3%. The performance and simplicity of this lightweight compression technique makes it an attractive option for coding an intermediate layer of a split neural network for edge/cloud applications.
Code (0)
등록된 구현이 없습니다.
Tasks
object-detectionObject DetectionQuantizationSimilar Papers 제목 키워드 기반
Lightweight compression of neural network feature tensors for collaborative intelligence
In collaborative intelligence applications, part of a deep neural network (DNN) is deployed on a relatively low-complexity device such as a mobile phone or edge device, and the remainder of the DNN is processed where mor…
object-detectionObject DetectionSNN-SC: A Spiking Semantic Communication Framework for Collaborative Intelligence
Collaborative Intelligence (CI) has emerged as a promising framework for deploying Artificial Intelligence (AI) models on resource-constrained edge devices. In CI, the AI model is partitioned between the edge device and …
ClassificationQuantizationSemantic CommunicationMulti-Agent Collaborative Inference via DNN Decoupling: Intermediate Feature Compression and Edge Learning
Recently, deploying deep neural network (DNN) models via collaborative inference, which splits a pre-trained model into two parts and executes them on user equipment (UE) and edge server respectively, becomes attractive.…
Collaborative InferenceCPUFeature CompressionRange Asymmetric Numeral Systems-Based Lightweight Intermediate Feature Compression for Split Computing of Deep Neural Networks
Split computing distributes deep neural network inference between resource-constrained edge devices and cloud servers but faces significant communication bottlenecks when transmitting intermediate features. To this end, …
Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme
Recently, a series of algorithms have been explored for GAN compression, which aims to reduce tremendous computational overhead and memory usages when deploying GANs on resource-constrained edge devices. However, most of…