QFT: Post-training quantization via fast joint finetuning of all degrees of freedom
The post-training quantization (PTQ) challenge of bringing quantized neural net accuracy close to original has drawn much attention driven by industry demand. Many of the methods emphasize optimization of a specific degree-of-freedom (DoF), such as quantization step size, preconditioning factors, bias fixing, often chained to others in multi-step solutions. Here we rethink quantized network parameterization in HW-aware fashion, towards a unified analysis of all quantization DoF, permitting for the first time their joint end-to-end finetuning. Our single-step simple and extendable method, dubbed quantization-aware finetuning (QFT), achieves 4-bit weight quantization results on-par with SoTA within PTQ constraints of speed and resource.
Code (0)
등록된 구현이 없습니다.
Tasks
AllQuantizationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
MatGPTQ: Accurate and Efficient Post-Training Matryoshka Quantization
Matryoshka Quantization (MatQuant) is a recent quantization approach showing that a single integer-quantized model can be served across multiple precisions, by slicing the most significant bits (MSB) at inference time. T…
Data-Free Group-Wise Fully Quantized Winograd Convolution via Learnable Scales
Despite the revolutionary breakthroughs of large-scale text-to-image diffusion models for complex vision and downstream tasks, their extremely high computational and storage costs limit their usability. Quantization of d…
image-classificationImage ClassificationImage GenerationQuantization+2Sensitivity-Aware Finetuning for Accuracy Recovery on Deep Learning Hardware
Existing methods to recover model accuracy on analog-digital hardware in the presence of quantization and analog noise include noise-injection training. However, it can be slow in practice, incurring high computational c…
Deep LearningQuantizationSensitivityLoss Aware Post-training Quantization
Neural network quantization enables the deployment of large models on resource-constrained devices. Current post-training quantization methods fall short in terms of accuracy for INT4 (or lower) but provide reasonable ac…
QuantizationOutlier-Aware Post-Training Quantization for Image Super-Resolution
Quantization techniques, including quantization-aware training (QAT) and post-training quantization (PTQ), have become essential for inference acceleration of image super-resolution (SR) networks. Compared to QAT, PTQ ha…
Image Super-Resolution