JacQuant: STE-Free Quantization-Aware Training via Learned Jacobian Surrogates

2026-05-25 · Kai Yi, Vignesh Vivekraja, Harshit Khaitan, Steven Li arxiv

Quantization-aware training (QAT) is widely deployed but typically relies on the Straight-Through Estimator (STE), which passes gradients through non-differentiable quantizers by fiat. This often makes training brittle near bin boundaries and weakly aligned with the actual behavior of the low-precision model. We introduce JacQuant, a QAT framework that learns a lightweight surrogate of the model's local sensitivity to parameter changes and uses it to stabilize and accelerate training within standard variance-reduced optimizers. The surrogate is inexpensive (diagonal or block-diagonal), data-driven, and compatible with common weight and activation quantizers. On code-preserving training phases, we prove convergence for non-convex objectives and obtain linear rates under a PL condition, and we relate the learned sensitivity to end-to-end output fidelity via a simple calibration argument. Across LLM benchmarks at $\leq 2$ bits, JacQuant consistently reaches higher accuracy than STE-based QAT, and the runtime analyses on various models show that the added cost remains negligible under practical group sizes. The method is drop-in and requires no changes to the forward quantizers; our empirical claims are scoped to ultra-low-bit LLM QAT.

📄 PDF Abstract BibTeX arXiv:2605.25469

Code (0)

등록된 구현이 없습니다.

LLM-QAT: Data-Free Quantization Aware Training for Large Language Models

2023-05-29 · Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang 외

Several post-training quantization methods have been applied to large language models (LLMs), and have been shown to perform well down to 8-bits. We find that these methods break down at lower bit precision, and investig…

Data Free Quantization Quantization

QFT: Post-training quantization via fast joint finetuning of all degrees of freedom

2022-12-05 · Alex Finkelstein, Ella Fuchs, Idan Tal, Mark Grobman 외

The post-training quantization (PTQ) challenge of bringing quantized neural net accuracy close to original has drawn much attention driven by industry demand. Many of the methods emphasize optimization of a specific degr…

All Quantization

EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models

2023-10-05 · Yefei He, Jing Liu, Weijia Wu, Hong Zhou 외

Diffusion models have demonstrated remarkable capabilities in image synthesis and related generative tasks. Nevertheless, their practicality for real-world applications is constrained by substantial computational costs a…

Denoising Image Generation parameter-efficient fine-tuning Quantization

DAQ: Delta-Aware Quantization for Post-Training LLM Weight Compression

2026-03-20 · Xiaoming Yu, Shize Tang, Guanghua Yu, Linchuan Xie 외 arxiv

We introduce Delta-Aware Quantization (DAQ), a data-free post-training quantization framework that preserves the knowledge acquired during post-training. Standard quantization objectives minimize reconstruction error but…

DAQ: Channel-Wise Distribution-Aware Quantization for Deep Image Super-Resolution Networks

2020-12-21 · Cheeun Hong, Heewon Kim, Sungyong Baik, Junghun Oh 외

Quantizing deep convolutional neural networks for image super-resolution substantially reduces their computational costs. However, existing works either suffer from a severe performance drop in ultra-low precision of 4 o…

Image Super-Resolution Quantization Super-Resolution