paper-with-me

Papers

Error Propagation Mechanisms and Compensation Strategies for Quantized Diffusion

2025-08-16 · Songwei Liu, Chao Zeng, Chenqian Yan, Xurui Peng, Xing Wang, Fangmin Chen, Xing Mei arxiv

Diffusion models have transformed image synthesis by establishing unprecedented quality and creativity benchmarks. Nevertheless, their large-scale deployment faces challenges due to computationally intensive iterative denoising processes. Although post-training quantization (PTQ) provides an effective pathway for accelerating sampling, the iterative nature of diffusion models causes stepwise quantization errors to accumulate progressively during generation, inevitably compromising output fidelity. To address this challenge, we develop a theoretical framework that mathematically formulates error propagation in Diffusion Models (DMs), deriving per-step quantization error propagation equations and establishing the first closed-form solution for cumulative error. Building on this theoretical foundation, we propose a timestep-aware cumulative error compensation scheme. Extensive experiments on multiple image datasets demonstrate that our compensation strategy effectively mitigates error propagation, significantly enhancing existing PTQ methods. Specifically, it achieves a 1.2 PSNR improvement over SVDQuant on SDXL W4A4, while incurring only an additional $<$ 0.5\% time overhead.

📄 PDF Abstract BibTeX arXiv:2508.12094

Code (0)

등록된 구현이 없습니다.

Similar Papers 제목 키워드 기반

Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models

2026-07-16 · Ryona Noda arxiv

Layer-wise post-training quantization of large language models minimizes each layer's reconstruction error in isolation, allowing quantization errors to accumulate across depth and causing severe degradation in extreme l…

Quantized Evolution Strategies: High-precision Fine-tuning of Quantized LLMs at Low-precision Cost

2026-02-03 · Yinggan Xu, Kajetan Schweighofer, Risto Miikkulainen, Xin Qiu arxiv

Post-Training Quantization (PTQ) is essential for deploying Large Language Models (LLMs) on memory-constrained devices, yet it renders models static and difficult to fine-tune. Standard fine-tuning paradigms, including R…

Reinforcement Learning

Quantitative Error Feedback for Quantization Noise Reduction of Filtering over Graphs

2025-06-02 · Xue Xian Zheng, Weihang Liu, Xin Lou, Stefan Vlaski 외

This paper introduces an innovative error feedback framework designed to mitigate quantization noise in distributed graph filtering, where communications are constrained to quantized messages. It comes from error spectru…

Quantization

Rethinking Residual Errors in Compensation-based LLM Quantization

2026-04-09 · Shuaiting Li, Juncan Deng, Kedong Xu, Rongtao Deng 외 arxiv

Methods based on weight compensation, which iteratively apply quantization and weight compensation to minimize the output error, have recently demonstrated remarkable success in quantizing Large Language Models (LLMs). T…

RILQ: Rank-Insensitive LoRA-based Quantization Error Compensation for Boosting 2-bit Large Language Model Accuracy

2024-12-02 · Geonho Lee, Janghwan Lee, Sukjin Hong, Minsoo Kim 외

Low-rank adaptation (LoRA) has become the dominant method for parameter-efficient LLM fine-tuning, with LoRA-based quantization error compensation (LQEC) emerging as a powerful tool for recovering accuracy in compressed …

Computational EfficiencyLanguage ModelingLanguage ModellingLarge Language Model+1