HQNAS: Auto CNN deployment framework for joint quantization and architecture search
Deep learning applications are being transferred from the cloud to edge with the rapid development of embedded computing systems. In order to achieve higher energy efficiency with the limited resource budget, neural networks(NNs) must be carefully designed in two steps, the architecture design and the quantization policy choice. Neural Architecture Search(NAS) and Quantization have been proposed separately when deploying NNs onto embedded devices. However, taking the two steps individually is time-consuming and leads to a sub-optimal final deployment. To this end, we propose a novel neural network design framework called Hardware-aware Quantized Neural Architecture Search(HQNAS) framework which combines the NAS and Quantization together in a very efficient manner using weight-sharing and bit-sharing. It takes only 4 GPU hours to discover an outstanding NN policy on CIFAR10. It also takes only %10 GPU time to generate a comparable model on Imagenet compared to the traditional NAS method with 1.8x decrease of latency and a negligible accuracy loss of only 0.7%. Besides, our method can be adapted in a lifelong situation where the neural network needs to evolve occasionally due to changes of local data, environment and user preference.
Code (0)
등록된 구현이 없습니다.
Tasks
GPUNeural Architecture SearchQuantizationSimilar Papers 제목 키워드 기반
Automated Model Compression by Jointly Applied Pruning and Quantization
In the traditional deep compression framework, iteratively performing network pruning and quantization can reduce the model size and computation cost to meet the deployment requirements. However, such a step-wise applica…
AutoMLModel CompressionNetwork Pruningobject-detection+2Towards Hardware-Specific Automatic Compression of Neural Networks
Compressing neural network architectures is important to allow the deployment of models to embedded or mobile devices, and pruning and quantization are the major approaches to compress neural networks nowadays. Both meth…
Quantizationreinforcement-learningReinforcement Learning (RL)Memory- and Latency-Constrained Inference of Large Language Models via Adaptive Split Computing
Large language models (LLMs) have achieved near-human performance across diverse reasoning tasks, yet their deployment on resource-constrained Internet-of-Things (IoT) devices remains impractical due to massive parameter…
E-PMQ: Expert-Guided Post-Merge Quantization with Merged-Weight Anchoring
Low-resource deployment constraints have made model quantization essential for deploying neural networks while preserving performance. Meanwhile, model merging has become an increasingly practical low-resource strategy f…
Joint Structural Pruning and Mixed-Precision Quantization for LLM Compression
Recently, the efficiency of Large Language Models (LLMs) deployment has become a critical concern in practical applications. While post-training quantization (PTQ) and structural pruning are established techniques for re…