Accelerating the Low-Rank Decomposed Models
Tensor decomposition is a mathematically supported technique for data compression. It consists of applying some kind of a Low Rank Decomposition technique on the tensors or matrices in order to reduce the redundancy of the data. However, it is not a popular technique for compressing the AI models duo to the high number of new layers added to the architecture after decomposition. Although the number of parameters could shrink significantly, it could result in the model be more than twice deeper which could add some latency to the training or inference. In this paper, we present a comprehensive study about how to modify low rank decomposition technique in AI models so that we could benefit from both high accuracy and low memory consumption as well as speeding up the training and inference
Code (0)
등록된 구현이 없습니다.
Tasks
Data CompressionTensor DecompositionSimilar Papers 제목 키워드 기반
Training Acceleration of Low-Rank Decomposed Networks using Sequential Freezing and Rank Quantization
Low Rank Decomposition (LRD) is a model compression technique applied to the weight tensors of deep learning models in order to reduce the number of trainable parameters and computational complexity. However, due to high…
Model CompressionQuantizationAutomatic Rank Selection for High-Speed Convolutional Neural Network
Low-rank decomposition plays a central role in accelerating convolutional neural network (CNN), and the rank of decomposed kernel-tensor is a key parameter that determines the complexity and accuracy of a neural network.…
Combinatorial OptimizationVocal Bursts Intensity PredictionAccelerating Training using Tensor Decomposition
Tensor decomposition is one of the well-known approaches to reduce the latency time and number of parameters of a pre-trained model. However, in this paper, we propose an approach to use tensor decomposition to reduce tr…
CPUGPUTensor DecompositionUniRank: Unified Rank Allocation for Low-Rank LLM Compression
Low-rank decomposition serves as a promising compression paradigm for large language models, however, rank allocation remains challenging: manual rules lack generalizability, and learning-based approaches incur heavy com…
Learning Low-Rank Approximation for CNNs
Low-rank approximation is an effective model compression technique to not only reduce parameter storage requirements, but to also reduce computations. For convolutional neural networks (CNNs), however, well-known low-ran…
Model Compression