paper-with-me

홈 › Papers

Panacea: Novel DNN Accelerator using Accuracy-Preserving Asymmetric Quantization and Energy-Saving Bit-Slice Sparsity

2024-12-13 · Dongyun Kam, Myeongji Yun, Sunwoo Yoo, Seungwoo Hong, Zhengya Zhang, Youngjoo Lee

Low bit-precisions and their bit-slice sparsity have recently been studied to accelerate general matrix-multiplications (GEMM) during large-scale deep neural network (DNN) inferences. While the conventional symmetric quantization facilitates low-resolution processing with bit-slice sparsity for both weight and activation, its accuracy loss caused by the activation's asymmetric distributions cannot be acceptable, especially for large-scale DNNs. In efforts to mitigate this accuracy loss, recent studies have actively utilized asymmetric quantization for activations without requiring additional operations. However, the cutting-edge asymmetric quantization produces numerous nonzero slices that cannot be compressed and skipped by recent bit-slice GEMM accelerators, naturally consuming more processing energy to handle the quantized DNN models. To simultaneously achieve high accuracy and hardware efficiency for large-scale DNN inferences, this paper proposes an Asymmetrically-Quantized bit-Slice GEMM (AQS-GEMM) for the first time. In contrast to the previous bit-slice computing, which only skips operations of zero slices, the AQS-GEMM compresses frequent nonzero slices, generated by asymmetric quantization, and skips their operations. To increase the slice-level sparsity of activations, we also introduce two algorithm-hardware co-optimization methods: a zero-point manipulation and a distribution-based bit-slicing. To support the proposed AQS-GEMM and optimizations at the hardware-level, we newly introduce a DNN accelerator, Panacea, which efficiently handles sparse/dense workloads of the tiled AQS-GEMM to increase data reuse and utilization. Panacea supports a specialized dataflow and run-length encoding to maximize data reuse and minimize external memory accesses, significantly improving its hardware efficiency. Our benchmark evaluations show Panacea outperforms existing DNN accelerators.

📄 PDF Abstract BibTeX arXiv:2412.10059

Code (0)

등록된 구현이 없습니다.

Tasks

Quantization

Similar Papers 제목 키워드 기반

Quant-Trim in Practice: Improved Cross-Platform Low-Bit Deployment on Edge NPUs

2025-11-19 · Rayen Dhahri, Steffen Urban arxiv

Specialized edge accelerators rely on low-bit quantization, but vendor compilers differ in scaling, clipping, and kernel support, often as black boxes. The same floating-point (FP) checkpoint can therefore yield inconsis…

GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inference

2026-07-30 · Sangjin Kim, Yuseon Choi, Byeongcheol Kim, Jungjun Oh 외 arxiv

Low-bit quantization is essential for efficient LLM inference, and both rotation and fine-grained group quantization have shown individual promise. However, their combination often leads to accuracy degradation or hardwa…

MRQ:Support Multiple Quantization Schemes through Model Re-Quantization

2023-08-01 · Manasa Manohara, Sankalp Dayal, Tariq Afzal, Rahul Bakshi 외

Despite the proliferation of diverse hardware accelerators (e.g., NPU, TPU, DPU), deploying deep learning models on edge devices with fixed-point hardware is still challenging due to complex model quantization and conver…

modelQuantization

Linear Symmetric Quantization of Neural Networks for Low-precision Integer Hardware

2020-05-01 · ICLR 2020 1 · Xiandong Zhao, Ying Wang, Xuyi Cai, Cheng Liu 외

With the proliferation of specialized neural network processors that operate on low-precision integers, the performance of Deep Neural Network inference becomes increasingly dependent on the result of quantization. Despi…

object-detectionObject DetectionQuantization

OPAL: Outlier-Preserved Microscaling Quantization Accelerator for Generative Large Language Models

2024-09-06 · Jahyun Koo, Dahoon Park, Sangwoo Jung, Jaeha Kung

To overcome the burden on the memory size and bandwidth due to ever-increasing size of large language models (LLMs), aggressive weight quantization has been recently studied, while lacking research on quantizing activati…

DecoderQuantization