paper-with-me

홈 › Papers

Geometry of Score Based Generative Models

2023-02-09 · Sandesh Ghimire, Jinyang Liu, Armand Comas, Davin Hill, Aria Masoomi, Octavia Camps, Jennifer Dy

In this work, we look at Score-based generative models (also called diffusion generative models) from a geometric perspective. From a new view point, we prove that both the forward and backward process of adding noise and generating from noise are Wasserstein gradient flow in the space of probability measures. We are the first to prove this connection. Our understanding of Score-based (and Diffusion) generative models have matured and become more complete by drawing ideas from different fields like Bayesian inference, control theory, stochastic differential equation and Schrodinger bridge. However, many open questions and challenges remain. One problem, for example, is how to decrease the sampling time? We demonstrate that looking from geometric perspective enables us to answer many of these questions and provide new interpretations to some known results. Furthermore, geometric perspective enables us to devise an intuitive geometric solution to the problem of faster sampling. By augmenting traditional score-based generative models with a projection step, we show that we can generate high quality images with significantly fewer sampling-steps.

📄 PDF Abstract BibTeX arXiv:2302.04411

Code (0)

등록된 구현이 없습니다.

Tasks

Bayesian Inference

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

Riemannian Score-Based Generative Modelling

2022-02-06 · Valentin De Bortoli, Emile Mathieu, Michael Hutchinson, James Thornton 외

Score-based generative models (SGMs) are a powerful class of generative models that exhibit remarkable empirical performance. Score-based generative modelling (SGM) consists of a ``noising'' stage, whereby a diffusion is…

Denoising

Intrinsic Wasserstein Rates for Score-Based Generative Models on Smooth Manifolds

2026-05-15 · Guoji Fu, Taiji Suzuki, Wee Sun Lee, Atsushi Nitanda arxiv

Score-based generative models are trained in high-dimensional ambient spaces, yet many data distributions are supported on low-dimensional nonlinear structures. We prove that, for compact $d$-dimensional smooth manifolds…

On the Anisotropy of Score-Based Generative Models

2025-10-27 · Andreas Floros, Seyed-Mohsen Moosavi-Dezfooli, Pier Luigi Dragotti arxiv

We investigate the role of network architecture in shaping the inductive biases of modern score-based generative models. To this end, we introduce the Score Anisotropy Directions (SADs), architecture-dependent directions…

Score-based Pullback Riemannian Geometry: Extracting the Data Manifold Geometry using Anisotropic Flows

2024-10-02 · Willem Diepeveen, Georgios Batzolis, Zakhar Shumaylov, Carola-Bibiane Schönlieb

Data-driven Riemannian geometry has emerged as a powerful tool for interpretable representation learning, offering improved efficiency in downstream tasks. Moving forward, it is crucial to balance cheap manifold mappings…

Representation Learning

Your Local GAN: Designing Two Dimensional Local Attention Mechanisms for Generative Models

2019-11-27 · CVPR 2020 6 · Giannis Daras, Augustus Odena, Han Zhang, Alexandros G. Dimakis

We introduce a new local sparse attention layer that preserves two-dimensional geometry and locality. We show that by just replacing the dense attention layer of SAGAN with our construction, we obtain very significant FI…

Conditional Image GenerationDeep AttentionImage Generation