Precision Highway for Ultra Low-Precision Quantization

2018-12-24 · ICLR 2019 5 · Eunhyeok Park, Dongyoung Kim, Sungjoo Yoo, Peter Vajda

Neural network quantization has an inherent problem called accumulated quantization error, which is the key obstacle towards ultra-low precision, e.g., 2- or 3-bit precision. To resolve this problem, we propose precision highway, which forms an end-to-end high-precision information flow while performing the ultra low-precision computation. First, we describe how the precision highway reduce the accumulated quantization error in both convolutional and recurrent neural networks. We also provide the quantitative analysis of the benefit of precision highway and evaluate the overhead on the state-of-the-art hardware accelerator. In the experiments, our proposed method outperforms the best existing quantization methods while offering 3-bit weight/activation quantization with no accuracy loss and 2-bit quantization with a 2.45 % top-1 accuracy loss in ResNet-50. We also report that the proposed method significantly outperforms the existing method in the 2-bit quantization of an LSTM for language modeling.

📄 PDF Abstract BibTeX arXiv:1812.09818

Code (0)

등록된 구현이 없습니다.

Tasks

Language ModelingLanguage Modelling Quantization

Methods 이 논문이 사용한 방법론

Sigmoid Activation 설명 없음

Tanh Activation 설명 없음

LSTM An LSTM is a type of recurrent neural network that addresses the vanishing gradient problem in vanilla…

Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuning

2023-07-02 · Jun Chen, Shipeng Bai, Tianxin Huang, Mengmeng Wang 외

Neural network quantization is a very promising solution in the field of model compression, but its resulting accuracy highly depends on a training/fine-tuning process and requires the original data. This not only brings…

Data Free Quantization Model Compression Quantization

DAQ: Channel-Wise Distribution-Aware Quantization for Deep Image Super-Resolution Networks

2020-12-21 · Cheeun Hong, Heewon Kim, Sungyong Baik, Junghun Oh 외

Quantizing deep convolutional neural networks for image super-resolution substantially reduces their computational costs. However, existing works either suffer from a severe performance drop in ultra-low precision of 4 o…

Image Super-Resolution Quantization Super-Resolution

DeepGEMM: Accelerated Ultra Low-Precision Inference on CPU Architectures using Lookup Tables

2023-04-18 · Darshan C. Ganji, Saad Ashfaq, Ehsan Saboori, Sudhakar Sah 외

A lot of recent progress has been made in ultra low-bit quantization, promising significant improvements in latency, memory footprint and energy consumption on edge devices. Quantization methods such as Learned Step Size…

CPU Quantization

WINDQuant: Weight-Informed Neural Decision-Making for Global Mixed-Precision LLM Quantization

2026-05-26 · Phong Nam Huu Nguyen, Khoi M. Le, Cong-Duy T Nguyen, Anh Tuan Luu 외 arxiv

Quantization is an effective approach to reduce the memory footprint and inference cost of large language models (LLMs), yet maintaining performance in the ultra-low-bit regime remains challenging. Existing post-training…

Reinforcement Learning

Automatic Network Adaptation for Ultra-Low Uniform-Precision Quantization

2022-12-21 · Seongmin Park, Beomseok Kwon, Jieun Lim, Kyuyoung Sim 외

Uniform-precision neural network quantization has gained popularity since it simplifies densely packed arithmetic unit for high computing capability. However, it ignores heterogeneous sensitivity to the impact of quantiz…

Neural Architecture Search Quantization