QFlash: Bridging Quantization and Memory Efficiency in Vision Transformer Attention

2026-04-28 · Sehyeon Oh, Yongin Kwon, Jemin Lee arxiv

FlashAttention improves efficiency through tiling, but its online softmax still relies on floating-point arithmetic for numerical stability, making full quantization difficult. We identify three main obstacles to integer-only FlashAttention: (1) scale explosion during tile-wise accumulation, (2) inefficient shift-based exponential operations on GPUs, and (3) quantization granularity constraints requiring uniform scales for integer comparison. To address these challenges, we propose \textit{QFlash}, an end-to-end integer FlashAttention design that performs softmax entirely in the integer domain and runs as a single Triton kernel. On seven attention workloads from ViT, DeiT, and Swin models, QFlash achieves up to 6.73$\times$ speedup over I-ViT and up to 8.69$\times$ speedup on Swin, while reducing energy consumption by 18.8\% compared to FP16 FlashAttention, without sacrificing Top-1 accuracy on ViT/DeiT and remaining competitive on Swin under per-tensor quantization. Our code is publicly available at https://github.com/EfficientCompLab/qflash.

📄 PDF Abstract BibTeX arXiv:2604.25306

Code (0)

등록된 구현이 없습니다.

QuartDepth: Post-Training Quantization for Real-Time Depth Estimation on the Edge

2025-03-20 · CVPR 2025 1 · Xuan Shen, Weize Ma, Jing Liu, Changdi Yang 외

Monocular Depth Estimation (MDE) has emerged as a pivotal task in computer vision, supporting numerous real-world applications. However, deploying accurate depth estimation models on resource-limited edge devices, especi…

Depth Estimation Monocular Depth Estimation Quantization

MLoRQ: Bridging Low-Rank and Quantization for Transformer Compression

2025-07-13 · Ofir Gordon, Ariel Lapid, Elad Cohen, Yarden Yagil 외 arxiv

Deploying transformer-based neural networks on resource-constrained edge devices presents a significant challenge. This challenge is often addressed through various techniques, such as low-rank approximation and mixed-pr…

Instance Segmentation Image Classification Object Detection

Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency

2025-04-04 · Erik Johannes Husom, Arda Goknil, Merve Astekin, Lwin Khin Shar 외

Deploying Large Language Models (LLMs) on edge devices presents significant challenges due to computational constraints, memory limitations, inference speed, and energy consumption. Model quantization has emerged as a ke…

Benchmarking GSM8K HumanEval Quantization+2

Scene-Aware Vectorized Memory Multi-Agent Framework with Cross-Modal Differentiated Quantization VLMs for Visually Impaired Assistance

2025-08-25 · Xiangxiang Wang, Xuanyu Wang, YiJia Luo, Yongbin Yu 외 arxiv

Visually impaired individuals face significant challenges in environmental perception. Traditional assistive technologies often lack adaptive intelligence, focusing on individual components rather than integrated systems…

Computational Efficiency

SQAP-VLA: A Synergistic Quantization-Aware Pruning Framework for High-Performance Vision-Language-Action Models

2025-09-11 · Hengyu Fang, Yijiang Liu, Yuan Du, Li Du 외 arxiv

Vision-Language-Action (VLA) models exhibit unprecedented capabilities for embodied intelligence. However, their extensive computational and memory costs hinder their practical deployment. Existing VLA compression and ac…