Technical Report: Activation Residual Hessian Quantization (ARHQ) for Low-Bit LLM Quantization

2026-04-30 · YiFeng Wang, Zhun Sun, Keisuke Sakaguchi arxiv

We present Activation Residual Hessian Quantization (ARHQ), a post-training weight splitting method designed to mitigate error propagation in low-bit activation-weight quantization. By constructing an input-side residual Hessian from activation quantization residuals (G_x), ARHQ analytically identifies and isolates error-sensitive weight directions into a high-precision low-rank branch. This is achieved via a closed-form truncated SVD on the scaled weight matrix W G^{1/2}_x . Experimental results on Qwen3-4B-Thinking-2507 demonstrate that ARHQ significantly improves layer-wise SNR and preserves downstream reasoning performance on ZebraLogic even under aggressive quantization. The code is available at https://github.com/BeautMoonQ/ARHQ.

📄 PDF Abstract BibTeX arXiv:2605.00140

Code (0)

등록된 구현이 없습니다.

Channel-wise Hessian Aware trace-Weighted Quantization of Neural Networks

2020-08-19 · Xu Qian, Victor Li, Crews Darren

Second-order information has proven to be very effective in determining the redundancy of neural network weights and activations. Recent paper proposes to use Hessian traces of weights and activations for mixed-precision…

AutoML Deep Reinforcement Learning Quantization

KronQ: LLM Quantization via Kronecker-Factored Hessian

2026-07-08 · Donghyun Lee, Yuhang Li, Ruokai Yin, Priyadarshini Panda arxiv

Post-training quantization (PTQ) is a widely adopted technique for compressing large language models (LLMs) without retraining. Existing second-order PTQ methods, including GPTQ, construct quantization objectives exclusi…

HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks

2019-11-10 · NeurIPS 2020 12 · Zhen Dong, Zhewei Yao, Yaohui Cai, Daiyaan Arfeen 외

Quantization is an effective method for reducing memory footprint and inference time of Neural Networks, e.g., for efficient inference in the cloud, especially at the edge. However, ultra low precision quantization could…

object-detection Object Detection Quantization Sensitivity

DynamicPTQ: Mitigating Activation Quantization Collapse via Residual-Stream Dynamics

2026-06-10 · Zimo Zhao, Maolin Wang, Bowen Yu, Bowen Liu 외 arxiv

Post-training quantization (PTQ) is essential for efficient large language model inference, but reliably quantizing activations remains challenging when weights, activations, and KV caches are all quantized to 4-bit prec…

On the Importance of a Multi-Scale Calibration for Quantization

2026-02-07 · Seungwoo Son, Ingyu Seong, Junhan Kim, Hyemi Jang 외 arxiv

Post-training quantization (PTQ) is a cornerstone for efficiently deploying large language models (LLMs), where a small calibration set critically affects quantization performance. However, conventional practices rely on…