Neural Networks with Quantization Constraints
Enabling low precision implementations of deep learning models, without considerable performance degradation, is necessary in resource and latency constrained settings. Moreover, exploiting the differences in sensitivity to quantization across layers can allow mixed precision implementations to achieve a considerably better computation performance trade-off. However, backpropagating through the quantization operation requires introducing gradient approximations, and choosing which layers to quantize is challenging for modern architectures due to the large search space. In this work, we present a constrained learning approach to quantization aware training. We formulate low precision supervised learning as a constrained optimization problem, and show that despite its non-convexity, the resulting problem is strongly dual and does away with gradient estimations. Furthermore, we show that dual variables indicate the sensitivity of the objective with respect to constraint perturbations. We demonstrate that the proposed approach exhibits competitive performance in image classification tasks, and leverage the sensitivity result to apply layer selective quantization based on the value of dual variables, leading to considerable performance improvements.
Code (1)
Tasks
image-classificationImage ClassificationQuantizationSensitivityMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
FracBits: Mixed Precision Quantization via Fractional Bit-Widths
Model quantization helps to reduce model size and latency of deep neural networks. Mixed precision quantization is favorable with customized hardwares supporting arithmetic operations at multiple bit-widths to achieve ma…
QuantizationHPTQ: Hardware-Friendly Post Training Quantization
Neural network quantization enables the deployment of models on edge devices. An essential requirement for their hardware efficiency is that the quantizers are hardware-friendly: uniform, symmetric, and with power-of-two…
object-detectionObject DetectionPose EstimationQuantization+1Fair-GPTQ: Bias-Aware Quantization for Large Language Models
The high memory demands of generative language models have drawn attention to quantization, which reduces memory usage by mapping model weights to lower-precision integers. However, recent empirical studies show that, wh…
Text GenerationStatic Quantized Radix-2 FFT/IFFT Processor for Constraints Analysis
This research work focuses on the design of a high-resolution fast Fourier transform (FFT) /inverse fast Fourier transform (IFFT) processors for constraints analysis purpose. Amongst the major setbacks associated with su…
QuantizationEffective and Fast: A Novel Sequential Single Path Search for Mixed-Precision Quantization
Since model quantization helps to reduce the model size and computation latency, it has been successfully applied in many applications of mobile phones, embedded devices and smart chips. The mixed-precision quantization …
Quantization