On-device AI: Quantization-aware Training of Transformers in Time-Series

2024-08-29 · Tianheng Ling, Gregor Schiele

Artificial Intelligence (AI) models for time-series in pervasive computing keep getting larger and more complicated. The Transformer model is by far the most compelling of these AI models. However, it is difficult to obtain the desired performance when deploying such a massive model on a sensor device with limited resources. My research focuses on optimizing the Transformer model for time-series forecasting tasks. The optimized model will be deployed as hardware accelerators on embedded Field Programmable Gate Arrays (FPGAs). I will investigate the impact of applying Quantization-aware Training to the Transformer model to reduce its size and runtime memory footprint while maximizing the advantages of FPGAs.

📄 PDF Abstract BibTeX arXiv:2408.16495

Code (0)

등록된 구현이 없습니다.

Tasks

QuantizationTime SeriesTime Series Forecasting

Methods 이 논문이 사용한 방법론

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Adam 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

Attention 설명 없음

Position-Wise Feed-Forward Layer 설명 없음

Dense Connections Dense Connections, or Fully Connected Connections, are a type of layer in a deep neural network that use a linear operation where every input is connected to every output…

Residual Connection 설명 없음

Multi-Head Attention 설명 없음

Quantization-Aware and Tensor-Compressed Training of Transformers for Natural Language Understanding

2023-06-01 · Zi Yang, Samridhi Choudhary, Siegfried Kunzmann, Zheng Zhang

Fine-tuned transformer models have shown superior performances in many natural language tasks. However, the large model size prohibits deploying high-performance transformer models on resource-constrained devices. This p…

Natural Language Understanding Quantization

UniQL: Unified Quantization and Low-rank Compression for Adaptive Edge LLMs

2025-12-03 · Hung-Yueh Chiang, Chi-Chih Chang, Yu-Chen Lu, Chien-Yu Lin 외 arxiv

Deploying large language models (LLMs) on mobile platforms faces significant challenges due to the limited memory and shared computational resources of the device. Resource availability may be an issue as it is directly …

Integer-only Quantized Transformers for Embedded FPGA-based Time-series Forecasting in AIoT

2024-07-06 · Tianheng Ling, Chao Qian, Gregor Schiele

This paper presents the design of a hardware accelerator for Transformers, optimized for on-device time-series forecasting in AIoT systems. It integrates integer-only quantization and Quantization-Aware Training with opt…

Quantization Time Series Time Series Forecasting

Resource-aware Mixed-precision Quantization for Enhancing Deployability of Transformers for Time-series Forecasting on Embedded FPGAs

2024-10-04 · Tianheng Ling, Chao Qian, Gregor Schiele

This study addresses the deployment challenges of integer-only quantized Transformers on resource-constrained embedded FPGAs (Xilinx Spartan-7 XC7S15). We enhanced the flexibility of our VHDL template by introducing a se…

Neural Architecture Search Quantization Time Series Time Series Forecasting

Understanding and Overcoming the Challenges of Efficient Transformer Quantization

2021-09-27 · EMNLP 2021 11 · Yelysei Bondarenko, Markus Nagel, Tijmen Blankevoort

Transformer-based architectures have become the de-facto standard models for a wide range of Natural Language Processing tasks. However, their memory footprint and high latency are prohibitive for efficient deployment an…

Quantization