Rotation Invariant Quantization for Model Compression

2023-03-03 · Joseph Kampeas, Yury Nahshan, Hanoch Kremer, Gil Lederman, Shira Zaloshinski, Zheng Li, Emir Haleva

Post-training Neural Network (NN) model compression is an attractive approach for deploying large, memory-consuming models on devices with limited memory resources. In this study, we investigate the rate-distortion tradeoff for NN model compression. First, we suggest a Rotation-Invariant Quantization (RIQ) technique that utilizes a single parameter to quantize the entire NN model, yielding a different rate at each layer, i.e., mixed-precision quantization. Then, we prove that our rotation-invariant approach is optimal in terms of compression. We rigorously evaluate RIQ and demonstrate its capabilities on various models and tasks. For example, RIQ facilitates $\times 19.4$ and $\times 52.9$ compression ratios on pre-trained VGG dense and pruned models, respectively, with $<0.4\%$ accuracy degradation. Code is available in \href{https://github.com/ehaleva/RIQ}{github.com/ehaleva/RIQ}.

📄 PDF Abstract BibTeX arXiv:2303.03106

Code (1)

ehaleva/riq 공식 구현 pytorch

Tasks

modelModel Compression Quantization

Methods 이 논문이 사용한 방법론

Max Pooling Max Pooling is a pooling operation that calculates the maximum value for patches of a feature map, and uses it to create a downsampled (pooled) feature map. It is usually…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Convolution A convolution is a type of matrix operation, consisting of a kernel, a small matrix of weights, that slides over input data performing element-wise multiplication with the…

Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…

RotateKV: Accurate and Robust 2-Bit KV Cache Quantization for LLMs via Outlier-Aware Adaptive Rotations

2025-01-25 · Zunhai Su, Zhe Chen, Wang Shen, Hanyu Wei 외

Key-Value (KV) cache facilitates efficient large language models (LLMs) inference by avoiding recomputation of past KVs. As the batch size and context length increase, the oversized KV caches become a significant memory …

Computational Efficiency GSM8K Quantization

SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving

2026-04-21 · Jinda Jia, Jisen Li, Zhongzhu Zhou, Jung Hwan Heo 외 arxiv

KV-cache memory is a major bottleneck in real-world LLM serving, where systems must simultaneously support latency-sensitive small-batch requests and high-throughput concurrent workloads. Although many KV-cache compressi…

PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compression

2026-03-30 · Caio Vicentino arxiv

We present PolarQuant, a post-training weight quantization method for large language models (LLMs) that exploits the distributional structure of neural network weights to achieve near-lossless compression. PolarQuant ope…

TR-DQ: Time-Rotation Diffusion Quantization

2025-03-09 · Yihua Shao, Deyang Lin, Fanhu Zeng, Minxi Yan 외

Diffusion models have been widely adopted in image and video generation. However, their complex network architecture leads to high inference overhead for its generation process. Existing diffusion quantization methods pr…

Image Generation Quantization Video Generation

Provable Quantization with Randomized Hadamard Transform

2026-05-13 · Ying Feng, Piotr Indyk, Michael Kapralov, Dmitry Krachun 외 arxiv

Vector quantization via random projection followed by scalar quantization is a fundamental primitive in machine learning, with applications ranging from similarity search to federated learning and KV cache compression. W…

Federated Learning