Activation Map Compression through Tensor Decomposition for Deep Learning
Internet of Things and Deep Learning are synergetically and exponentially growing industrial fields with a massive call for their unification into a common framework called Edge AI. While on-device inference is a well-explored topic in recent research, backpropagation remains an open challenge due to its prohibitive computational and memory costs compared to the extreme resource constraints of embedded devices. Drawing on tensor decomposition research, we tackle the main bottleneck of backpropagation, namely the memory footprint of activation map storage. We investigate and compare the effects of activation compression using Singular Value Decomposition and its tensor variant, High-Order Singular Value Decomposition. The application of low-order decomposition results in considerable memory savings while preserving the features essential for learning, and also offers theoretical guarantees to convergence. Experimental results obtained on main-stream architectures and tasks demonstrate Pareto-superiority over other state-of-the-art solutions, in terms of the trade-off between generalization and memory footprint.
Code (1)
Tasks
Deep LearningTensor DecompositionSimilar Papers 제목 키워드 기반
ESPACE: Dimensionality Reduction of Activations for Model Compression
We propose ESPACE, an LLM compression technique based on dimensionality reduction of activations. Unlike prior works on weight-centric tensor decomposition, ESPACE projects activations onto a pre-calibrated set of princi…
Dimensionality ReductionmodelModel CompressionTensor DecompositionTensor Shape Search for Optimum Data Compression
Various tensor decomposition methods have been proposed for data compression. In real world applications of the tensor decomposition, selecting the tensor shape for the given data poses a challenge and the shape of the t…
Data CompressionTensor DecompositionLatentLLM: Attention-Aware Joint Tensor Compression
Modern foundation models such as large language models (LLMs) and large multi-modal models (LMMs) require a massive amount of computational and memory resources. We propose a new framework to convert such LLMs/LMMs into …
Model CompressionTensor DecompositionSemi-tensor Product-based TensorDecomposition for Neural Network Compression
The existing tensor networks adopt conventional matrix product for connection. The classical matrix product requires strict dimensionality consistency between factors, which can result in redundancy in data representatio…
Low-rank compressionNeural Network CompressionTensor NetworksLow-Rank+Sparse Tensor Compression for Neural Networks
Low-rank tensor compression has been proposed as a promising approach to reduce the memory and compute requirements of neural networks for their deployment on edge devices. Tensor compression reduces the number of parame…
Tensor Decomposition