Block Rotation is All You Need for MXFP4 Quantization

2025-11-06 · Yuantian Shao, Peisong Wang, Yuanteng Chen, Chang Xu, Zhihui Wei, Jian Cheng arxiv

Large language models (LLMs) have achieved remarkable success, but their rapidly growing scale imposes prohibitive costs in memory, computation, and energy. Post-training quantization (PTQ) is a promising solution for efficient deployment, yet achieving accurate W4A4 quantization remains an open challenge. While most existing methods are designed for INT4 formats, the emergence of MXFP4 -- a new FP4 format with various hardware support (NVIDIA, AMD, Intel)-- raises questions about the applicability of current techniques. In this work, we establish a comprehensive benchmark of PTQ methods under the MXFP4 format. Through systematic evaluation, we find that methods like GPTQ consistently deliver strong performance, whereas rotation-based approaches, which are almost used by all state-of-the-art approaches, suffer from severe incompatibility with MXFP4. We further provide the first in-depth analysis of this conflict, tracing its root to a fundamental mismatch between MXFP4's PoT (power-of-two) block scaling and the redistribution of outlier energy via global rotation. Building on this insight, we propose a simple yet effective block rotation strategy that adapts rotation-based methods to MXFP4, leading to substantial accuracy improvements across diverse LLMs. Our findings not only offer clear guidance for practitioners but also set a foundation for advancing PTQ research under emerging low-precision formats.

📄 PDF Abstract BibTeX arXiv:2511.04214

Code (0)

등록된 구현이 없습니다.

TORQ: Two-Level Orthogonal Rotation for MXFP4 Quantization

2026-05-19 · Zukang Xu, Xing Hu, Dawei Yang arxiv

As Large Language Models (LLMs) advance toward practical deployment, the Microscaling FP4 (MXFP4) format has emerged as a cornerstone for next-generation low-bit inference, owing to its ability to balance high dynamic ra…

DuQuant++: Fine-grained Rotation Enhances Microscaling FP4 Quantization

2026-04-20 · Haokun Lin, Xinle Jia, Haobo Xu, Bingchen Yao 외 arxiv

The MXFP4 microscaling format, which partitions tensors into blocks of 32 elements sharing an E8M0 scaling factor, has emerged as a promising substrate for efficient LLM inference, backed by native hardware support on NV…

BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimization

2026-03-17 · Ji-Fu Li, Manyi Zhang, Xiaobo Xia, Han Bao 외 arxiv

Microscaling floating-point (MXFP) formats have emerged as a promising standard for deploying Multi-modal Large Language Models (MLLMs) and Large Language Models (LLMs) on modern accelerator architectures. However, exist…

AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference

2024-11-15 · Janghwan Lee, Jiwoong Park, Jinseok Kim, Yongjik Kim 외

Scaling Large Language Models (LLMs) with extended context lengths has increased the need for efficient low-bit quantization to manage their substantial computational demands. However, reducing precision to 4 bits freque…

Quantization Question Answering Visual Question Answering

Unveiling the Potential of Quantization with MXFP4: Strategies for Quantization Error Reduction

2026-01-30 · Jatin Chhugani, Geonhwa Jeong, Bor-Yiing Su, Yunjie Pan 외 arxiv

Large Language Models (LLMs) have intensified the need for low-precision formats that enable efficient, large-scale inference. The Open Compute Project (OCP) Microscaling (MX) standard is attractive due to its favorable …