paper-with-me

Papers

Provable Acceleration for Diffusion Models under Minimal Assumptions

2024-10-30 · Gen Li, Changxiao Cai

Score-based diffusion models, while achieving minimax optimality for sampling, are often hampered by slow sampling speeds due to the high computational burden of score function evaluations. Despite the recent remarkable empirical advances in speeding up the score-based samplers, theoretical understanding of acceleration techniques remains largely limited. To bridge this gap, we propose a novel training-free acceleration scheme for stochastic samplers. Under minimal assumptions -- namely, $L^2$-accurate score estimates and a finite second-moment condition on the target distribution -- our accelerated sampler provably achieves $\varepsilon$-accuracy in total variation within $\widetilde{O}(d^{5/4}/\sqrt{\varepsilon})$ iterations, thereby significantly improving upon the $\widetilde{O}(d/\varepsilon)$ iteration complexity of standard score-based samplers for $\varepsilon\leq 1/\sqrt{d}$. Notably, our convergence theory does not rely on restrictive assumptions on the target distribution or higher-order score estimation guarantees.

📄 PDF Abstract BibTeX arXiv:2410.23285

Code (0)

등록된 구현이 없습니다.

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

Rethinking SIGN Training: Provable Nonconvex Acceleration without First- and Second-Order Gradient Lipschitz

2023-10-23 · Tao Sun, Congliang Chen, Peng Qiao, Li Shen 외

Sign-based stochastic methods have gained attention due to their ability to achieve robust performance despite using only the sign information for parameter updates. However, the current convergence analysis of sign-base…

Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Models

2024-10-07 · Yuchen Wu, Yuxin Chen, Yuting Wei

Diffusion models play a pivotal role in contemporary generative modeling, claiming state-of-the-art performance across various domains. Despite their superior sample quality, mainstream diffusion-based stochastic sampler…

Multi-Step Consistency Models: Fast Generation with Theoretical Guarantees

2025-05-02 · Nishant Jain, Xunpeng Huang, Yian Ma, Tong Zhang

Consistency models have recently emerged as a compelling alternative to traditional SDE-based diffusion models. They offer a significant acceleration in generation by producing high-quality samples in very few steps. Des…

Provably convergent acceleration in factored gradient descent with applications in matrix sensing

2018-06-01 · Tayo Ajayi, David Mildebrath, Anastasios Kyrillidis, Shashanka Ubaru 외

We present theoretical results on the convergence of \emph{non-convex} accelerated gradient descent in matrix factorization models with $\ell_2$-norm loss. The purpose of this work is to study the effects of acceleration…

Quantum State Tomography

IHT dies hard: Provable accelerated Iterative Hard Thresholding

2017-12-26 · Rajiv Khanna, Anastasios Kyrillidis

We study --both in theory and practice-- the use of momentum motions in classic iterative hard thresholding (IHT) methods. By simply modifying plain IHT, we investigate its convergence behavior on convex optimization cri…