Compression-aware Projection with Greedy Dimension Reduction for Convolutional Neural Network Activations
Convolutional neural networks (CNNs) achieve remarkable performance in a wide range of fields. However, intensive memory access of activations introduces considerable energy consumption, impeding deployment of CNNs on resourceconstrained edge devices. Existing works in activation compression propose to transform feature maps for higher compressibility, thus enabling dimension reduction. Nevertheless, in the case of aggressive dimension reduction, these methods lead to severe accuracy drop. To improve the trade-off between classification accuracy and compression ratio, we propose a compression-aware projection system, which employs a learnable projection to compensate for the reconstruction loss. In addition, a greedy selection metric is introduced to optimize the layer-wise compression ratio allocation by considering both accuracy and #bits reduction simultaneously. Our test results show that the proposed methods effectively reduce 2.91x~5.97x memory access with negligible accuracy drop on MobileNetV2/ResNet18/VGG16.
Code (0)
등록된 구현이 없습니다.
Tasks
Dimensionality ReductionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
SHOE: Supervised Hashing with Output Embeddings
We present a supervised binary encoding scheme for image retrieval that learns projections by taking into account similarity between classes obtained from output embeddings. Our motivation is that binary hash codes learn…
AttributeDimensionality ReductionImage RetrievalRetrieval+1Reduced Basis Decomposition: a Certified and Fast Lossy Data Compression Algorithm
Dimension reduction is often needed in the area of data mining. The goal of these methods is to map the given high-dimensional data into a low-dimensional space preserving certain properties of the initial data. There ar…
Data CompressionDimensionality ReductionWavelet Based Dictionaries for Piecewise Dimensionality Reduction of ECG Signals
Dimensionality reduction of ECG signals is considered within the framework of sparse representation. The approach constructs the signal model by selecting elementary components from a redundant dictionary via a greedy st…
Dimensionality ReductionRandom Projections for Manifold Learning
We propose a novel method for {\em linear} dimensionality reduction of manifold modeled data. First, we show that with a small number $M$ of {\em random projections} of sample points in $\reals^N$ belonging to an unknown…
Dimensionality ReductionSTAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Control
Low-rank projection has emerged as a promising approach for compressing the KV cache by exploiting hidden-dimension redundancy. However, prior methods rely on fixed or heuristic rank selection and struggle to achieve agg…