AdaBits: Neural Network Quantization with Adaptive Bit-Widths
Deep neural networks with adaptive configurations have gained increasing attention due to the instant and flexible deployment of these models on platforms with different resource budgets. In this paper, we investigate a novel option to achieve this goal by enabling adaptive bit-widths of weights and activations in the model. We first examine the benefits and challenges of training quantized model with adaptive bit-widths, and then experiment with several approaches including direct adaptation, progressive training and joint training. We discover that joint training is able to produce comparable performance on the adaptive model as individual models. We further propose a new technique named Switchable Clipping Level (S-CL) to further improve quantized models at the lowest bit-width. With our proposed techniques applied on a bunch of models including MobileNet-V1/V2 and ResNet-50, we demonstrate that bit-width of weights and activations is a new option for adaptively executable deep neural networks, offering a distinct opportunity for improved accuracy-efficiency trade-off as well as instant adaptation according to the platform constraints in real-world applications.
Code (1)
Tasks
QuantizationSimilar Papers 제목 키워드 기반
Bitwidth-Adaptive Quantization-Aware Neural Network Training: A Meta-Learning Approach
Deep neural network quantization with adaptive bitwidths has gained increasing attention due to the ease of model deployment on various platforms with different resource budgets. In this paper, we propose a meta-learning…
Few-Shot LearningMeta-LearningQuantizationArbitrary Bit-width Network: A Joint Layer-Wise Quantization and Adaptive Inference Approach
Conventional model quantization methods use a fixed quantization scheme to different data samples, which ignores the inherent "recognition difficulty" differences between various samples. We propose to feed different dat…
QuantizationAdaQAT: Adaptive Bit-Width Quantization-Aware Training
Large-scale deep neural networks (DNNs) have achieved remarkable success in many application scenarios. However, high computational complexity and energy costs of modern DNNs make their deployment on edge devices challen…
QuantizationMixQuant: Adaptive Mixed-Precision Quantization for Large Language Models
Mixed-precision quantization improves the accuracy of post-training quantization by allocating higher bitwidths to sensitive layers, but existing methods solve the allocation for a single fixed memory budget. In practice…
Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths
Quantizing deep networks with adaptive bit-widths is a promising technique for efficient inference across many devices and resource constraints. In contrast to static methods that repeat the quantization process and trai…
image-classificationImage ClassificationQuantizationTransfer Learning+1