COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization

2024-03-11 · Aozhong zhang, Zi Yang, Naigang Wang, Yingyong Qi, Jack Xin, Xin Li, Penghang Yin

Post-training quantization (PTQ) has emerged as a practical approach to compress large neural networks, making them highly efficient for deployment. However, effectively reducing these models to their low-bit counterparts without compromising the original accuracy remains a key challenge. In this paper, we propose an innovative PTQ algorithm termed COMQ, which sequentially conducts coordinate-wise minimization of the layer-wise reconstruction errors. We consider the widely used integer quantization, where every quantized weight can be decomposed into a shared floating-point scalar and an integer bit-code. Within a fixed layer, COMQ treats all the scaling factor(s) and bit-codes as the variables of the reconstruction error. Every iteration improves this error along a single coordinate while keeping all other variables constant. COMQ is easy to use and requires no hyper-parameter tuning. It instead involves only dot products and rounding operations. We update these variables in a carefully designed greedy order, significantly enhancing the accuracy. COMQ achieves remarkable results in quantizing 4-bit Vision Transformers, with a negligible loss of less than 1% in Top-1 accuracy. In 4-bit INT quantization of convolutional neural networks, COMQ maintains near-lossless accuracy with a minimal drop of merely 0.3% in Top-1 accuracy.

📄 PDF Abstract BibTeX arXiv:2403.07134

Code (1)

aozhongzhang/comq 공식 구현 pytorch

Tasks

Quantization

Weight Group-wise Post-Training Quantization for Medical Foundation Model

2026-04-09 · Yineng Chen, Peng Huang, Aozhong Zhang, Hui Guo 외 arxiv

Foundation models have achieved remarkable results in medical image analysis. However, its large network architecture and high computational complexity significantly impact inference speed, limiting its application on te…

ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusters

2018-09-25 · NAACL 2019 6 · Abdalghani Abujabal, Rishiraj Saha Roy, Mohamed Yahya, Gerhard Weikum

To bridge the gap between the capabilities of the state-of-the-art in factoid question answering (QA) and what users ask, we need large datasets of real user questions that capture the various question phenomena users ar…

Question Answering

ChineseEcomQA: A Scalable E-commerce Concept Evaluation Benchmark for Large Language Models

2025-02-27 · Haibin Chen, Kangtao Lv, Chengwei Hu, Yanshi Li 외

With the increasing use of Large Language Models (LLMs) in fields such as e-commerce, domain-specific concept evaluation benchmarks are crucial for assessing their domain capabilities. Existing LLMs may generate factuall…

Question Answering RAG Retrieval-augmented Generation Specificity

Attention-aware Post-training Quantization without Backpropagation

2024-06-19 · Junhan Kim, Ho-young Kim, Eulrang Cho, Chungman Lee 외

Quantization is a promising solution for deploying large-scale language models (LLMs) on resource-constrained devices. Existing quantization approaches, however, rely on gradient-based optimization, regardless of it bein…