Discrete vs. Continuous Trade-offs for Generative Models
This work explores the theoretical and practical foundations of denoising diffusion probabilistic models (DDPMs) and score-based generative models, which leverage stochastic processes and Brownian motion to model complex data distributions. These models employ forward and reverse diffusion processes defined through stochastic differential equations (SDEs) to iteratively add and remove noise, enabling high-quality data generation. By analyzing the performance bounds of these models, we demonstrate how score estimation errors propagate through the reverse process and bound the total variation distance using discrete Girsanov transformations, Pinsker's inequality, and the data processing inequality (DPI) for an information theoretic lens.
Code (0)
등록된 구현이 없습니다.
Tasks
DenoisingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Discrete-state Continuous-time Diffusion for Graph Generation
Graph is a prevalent discrete data structure, whose generation has wide applications such as drug discovery and circuit design. Diffusion generative models, as an emerging research focus, have been applied to graph gener…
Drug DiscoveryGraph GenerationKnowledge Compilation with Continuous Random Variables and its Application in Hybrid Probabilistic Logic Programming
In probabilistic reasoning, the traditionally discrete domain has been elevated to the hybrid domain encompassing additionally continuous random variables. Inference in the hybrid domain, however, usually necessitates to…
Discrete Diffusion Models for Language Generation
Diffusion models have emerged as a powerful class of generative models, achieving state-of-the-art results in continuous data domains such as image and video generation. Their core mechanism involves a forward diffusion …
Video GenerationInformed Correctors for Discrete Diffusion Models
Discrete diffusion modeling is a promising framework for modeling and generating data in discrete spaces. To sample from these models, different strategies present trade-offs between computation and sample quality. A pre…
Simplified Analysis on Filtering Sensitivity Trade-offs in Continuous- and Discrete-Time Systems
A simplified analysis is performed on the Bode-type filtering sensitivity trade-off integrals, which capture the sensitivity characteristics of the estimate and estimation error with respect to the process input and esti…
Sensitivity