Once Quantized for All: Progressively Searching for Quantized Compact Models
Automatic search of Quantized Neural Networks (QNN) has attracted a lot of attention. However, the existing quantization-aware Neural Architecture Search (NAS) approaches inherit a two-stage search-retrain schema, which is not only time-consuming but also adversely affected by the unreliable ranking of architectures during the search. To avoid the undesirable effect of the search-retrain schema, we present Once Quantized for All (OQA), a novel framework that searches for quantized compact models and deploys their quantized weights at the same time without additional post-process. While supporting a huge architecture search space, our OQA can produce a series of quantized compact models under ultra-low bit-widths(e.g. 4/3/2 bit). A progressive bit inheritance procedure is introduced to support ultra-low bit-width. Our searched model family, OQANets, achieves a new state-of-the-art (SOTA) on quantized compact models compared with various quantization methods and bit-widths. In particular, OQA2bit-L achieves 64.0\% ImageNet Top-1 accuracy, outperforming its 2 bit counterpart EfficientNet-B0@QKD by a large margin of 14\% using 30\% less computation cost.
Code (0)
등록된 구현이 없습니다.
Tasks
AllNeural Architecture SearchQuantizationSimilar Papers 제목 키워드 기반
Vertical Layering of Quantized Neural Networks for Heterogeneous Inference
Although considerable progress has been obtained in neural network quantization for efficient inference, existing methods are not scalable to heterogeneous devices as one dedicated model needs to be trained, transmitted,…
QuantizationQUEST: Quantized embedding space for transferring knowledge
Knowledge distillation refers to the process of training a compact student network to achieve better accuracy by learning from a high capacity teacher network. Most of the existing knowledge distillation methods direct t…
Knowledge DistillationSearching for Low-Bit Weights in Quantized Neural Networks
Quantized neural networks with low-bit weights and activations are attractive for developing AI accelerators. However, the quantization functions used in most conventional quantization methods are non-differentiable, whi…
image-classificationImage ClassificationQuantizationSuper-ResolutionTowards Accurate Quantization and Pruning via Data-free Knowledge Transfer
When large scale training data is available, one can obtain compact and accurate networks to be deployed in resource-constrained environments effectively through quantization and pruning. However, training data are often…
Data Free QuantizationQuantizationTransfer LearningCollaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-width Deep Neural Networks
Knowledge distillation which learns a lightweight student model by distilling knowledge from a cumbersome teacher model is an attractive approach for learning compact deep neural networks (DNNs). Recent works further imp…
Knowledge DistillationQuantization