Frequency Regulation for Exposure Bias Mitigation in Diffusion Models
Diffusion models exhibit impressive generative capabilities but are significantly impacted by exposure bias. In this paper, we make a key observation: the energy of predicted noisy samples in the reverse process continuously declines compared to perturbed samples in the forward process. Building on this, we identify two important findings: 1) The reduction in energy follows distinct patterns in the low-frequency and high-frequency subbands; 2) The subband energy of reverse-process reconstructed samples is consistently lower than that of forward-process ones, and both are lower than the original data samples. Based on the first finding, we introduce a dynamic frequency regulation mechanism utilizing wavelet transforms, which separately adjusts the low- and high-frequency subbands. Leveraging the second insight, we derive the rigorous mathematical form of exposure bias. It is worth noting that, our method is training-free and plug-and-play, significantly improving the generative quality of various diffusion models and frameworks with negligible computational cost. The source code is available at https://github.com/kunzhan/wpp.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Exposure Bias Can Alleviate Itself via Directional and Frequency Rectification in Flow Matching
Flow Matching (FM) has achieved remarkable generative performance, yet it suffers from exposure bias due to discrepancies between training and inference. Existing mitigation strategies typically rely on static constraint…
Bias mitigation in graph diffusion models
Most existing graph diffusion models have significant bias problems. We observe that the forward diffusion's maximum perturbation distribution in most models deviates from the standard Gaussian distribution, while revers…
Health Risks Associated with 5G Exposure: A View from the Communications Engineering Perspective
The deployment of 5G wireless communication services requires the installation of 5G next-generation Node-B Base Stations (gNBs) over the territory and the wide adoption of 5G User Equipment (UE). In this context, the po…
Spectral Prior for Reducing Exposure Bias in Diffusion Models
Diffusion models typically suffer from error accumulation during iterative sampling, commonly referred to as exposure bias. We reveal systematic frequency-dependent discrepancies between training and inference, which can…
Data-based optimal estimation of frequency bias: The case of Southwest Power Pool
In this paper, we introduce a method to optimally estimate time-varying frequency bias. Current industry practice is to assume that frequency bias is changing only on annual basis. We suggest that this improved time-depe…
parameter estimation