Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning
Large language models have achieved significant advancements in complex mathematical reasoning benchmarks, such as MATH. However, their substantial computational requirements present challenges for practical deployment. Model quantization has emerged as an effective strategy to reduce memory usage and computational costs by employing lower precision and bit-width representations. In this study, we systematically evaluate the impact of quantization on mathematical reasoning tasks. We introduce a multidimensional evaluation framework that qualitatively assesses specific capability dimensions and conduct quantitative analyses on the step-by-step outputs of various quantization methods. Our results demonstrate that quantization differentially affects numerical computation and reasoning planning abilities, identifying key areas where quantized models experience performance degradation.
Code (0)
등록된 구현이 없습니다.
Tasks
MathMathematical ReasoningQuantizationSimilar Papers 제목 키워드 기반
Exploring Neural Networks Quantization via Layer-Wise Quantization Analysis
Quantization is an essential step in the efficient deployment of deep learning models and as such is an increasingly popular research topic. An important practical aspect that is not addressed in the current literature i…
QuantizationMixKVQ: Query-Aware Mixed-Precision KV Cache Quantization for Long-Context Reasoning
Long Chain-of-Thought (CoT) reasoning has significantly advanced the capabilities of Large Language Models (LLMs), but this progress is accompanied by substantial memory and latency overhead from the extensive Key-Value …
ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inference
Post-training quantization (PTQ) compresses the weights and activations of large language models (LLMs) into low-precision representations to reduce memory footprint and accelerate inference. However, the presence of out…
R2Q: Towards Robust 2-Bit Large Language Models via Residual Refinement Quantization
The rapid progress of Large Language Models (LLMs) has brought substantial computational and memory demands, spurring the adoption of low-bit quantization. While 8-bit and 4-bit formats have become prevalent, extending q…
Question AnsweringDo Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study
Despite the superior performance, Large Language Models~(LLMs) require significant computational resources for deployment and use. To overcome this issue, quantization methods have been widely applied to reduce the memor…
In-Context LearningInstruction FollowingQuantization