홈 › Papers

DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization

2026-04-20 · Haokun Lin, Xinle Jia, Haobo Xu, Bingchen Yao, Xianglong Guo, Yichen Wu, Zhichao Lu, Ying Wei, Qingfu Zhang, Zhenan Sun arxiv

The MXFP4 microscaling format, which partitions tensors into blocks of 32 elements sharing an E8M0 scaling factor, has emerged as a promising substrate for efficient LLM inference, backed by native hardware support on NVIDIA Blackwell Tensor Cores. However, activation outliers pose a unique challenge under this format: a single outlier inflates the shared block scale, compressing the effective dynamic range of the remaining elements and causing significant quantization error. Existing rotation-based remedies, including randomized Hadamard and learnable rotations, are data-agnostic and therefore unable to specifically target the channels where outliers concentrate. We propose DuQuant++, which adapts the outlier-aware fine-grained rotation of DuQuant to the MXFP4 format by aligning the rotation block size with the microscaling group size (B{=}32). Because each MXFP4 group possesses an independent scaling factor, the cross-block variance issue that necessitates dual rotations and a zigzag permutation in the original DuQuant becomes irrelevant, enabling DuQuant++ to replace the entire pipeline with a single outlier-aware rotation, which halves the online rotation cost while simultaneously smoothing the weight distribution. Extensive experiments on the LLaMA-3 family under MXFP4 W4A4 quantization show that DuQuant++ consistently achieves state-of-the-art performance. Our code is available at https://github.com/Hsu1023/DuQuant-v2.

📄 PDF Abstract BibTeX arXiv:2604.17789

Code (0)

등록된 구현이 없습니다.

DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs

2024-06-03 · Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui 외

Quantization of large language models (LLMs) faces significant challenges, particularly due to the presence of outlier activations that impede efficient low-bit representation. Traditional approaches predominantly addres…

Management Quantization

LATMiX: Learnable Affine Transformations for Microscaling Quantization of LLMs

2026-02-04 · Ofir Gordon, Lior Dikstein, Arnon Netzer, Idan Achituve 외 arxiv

Post-training quantization (PTQ) is a widely used approach for reducing the memory and compute costs of large language models (LLMs). Recent studies have shown that applying invertible transformations to activations can …

AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference

2024-11-15 · Janghwan Lee, Jiwoong Park, Jinseok Kim, Yongjik Kim 외

Scaling Large Language Models (LLMs) with extended context lengths has increased the need for efficient low-bit quantization to manage their substantial computational demands. However, reducing precision to 4 bits freque…

Quantization Question Answering Visual Question Answering

Microscaling Floating Point Formats for Large Language Models

2025-10-02 · Marco Cococcioni, Dario Pagani, Federico Rossi arxiv

The increasing computational and memory demands of large language models (LLMs) necessitate innovative approaches to optimize resource usage without compromising performance. This paper leverages microscaling floating-po…

Rethinking Rotation-Invariant Recognition of Fine-grained Shapes from the Perspective of Contour Points

2025-03-14 · Yanjie Xu, Handing Xu, Tianmu Wang, Yaguan Li 외

Rotation-invariant recognition of shapes is a common challenge in computer vision. Recent approaches have significantly improved the accuracy of rotation-invariant recognition by encoding the rotational invariance of sha…