Target Score Matching

2024-02-13 · Valentin De Bortoli, Michael Hutchinson, Peter Wirnsberger, Arnaud Doucet

Denoising Score Matching estimates the score of a noised version of a target distribution by minimizing a regression loss and is widely used to train the popular class of Denoising Diffusion Models. A well known limitation of Denoising Score Matching, however, is that it yields poor estimates of the score at low noise levels. This issue is particularly unfavourable for problems in the physical sciences and for Monte Carlo sampling tasks for which the score of the clean original target is known. Intuitively, estimating the score of a slightly noised version of the target should be a simple task in such cases. In this paper, we address this shortcoming and show that it is indeed possible to leverage knowledge of the target score. We present a Target Score Identity and corresponding Target Score Matching regression loss which allows us to obtain score estimates admitting favourable properties at low noise levels.

📄 PDF Abstract BibTeX arXiv:2402.08667

Code (0)

등록된 구현이 없습니다.

Tasks

Denoising regression

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Denoising Score Matching Training a denoiser on signals gives you a powerful prior over this signal that you can then use to sample examples of this signal.

Latent Target Score Matching, with an application to Simulation-Based Inference

2026-02-06 · Joohwan Ko, Tomas Geffner arxiv

Denoising score matching (DSM) for training diffusion models may suffer from high variance at low noise levels. Target Score Matching (TSM) mitigates this when clean data scores are available, providing a low-variance ob…

ScoreMatchingRiesz: Score Matching for Debiased Machine Learning and Policy Path Estimation

2025-12-23 · Masahiro Kato arxiv

We propose ScoreMatchingRiesz, a family of Riesz representer estimators based on score matching. The Riesz representer is a key nuisance component in debiased machine learning, enabling $\sqrt{n}$-consistent and asymptot…

Variational Inference with Gaussian Score Matching

2023-07-15 · NeurIPS 2023 11 · Chirag Modi, Charles Margossian, Yuling Yao, Robert Gower 외

Variational inference (VI) is a method to approximate the computationally intractable posterior distributions that arise in Bayesian statistics. Typically, VI fits a simple parametric distribution to the target posterior…

Bayesian Inference Variational Inference

Diffusion Models Observe Only Gradients: A Geometric Perspective on Score Matching Errors

2026-06-04 · Naïl B. Khelifa, Richard E. Turner, Ramji Venkataramanan arxiv

Score-based diffusion models are typically trained by minimizing the $L^2$ score matching error, and standard theoretical analyses rely on this quantity to bound the sampling discrepancy between the learned and target di…

A unified perspective on fine-tuning and sampling with diffusion and flow models

2026-04-30 · Carles Domingo-Enrich, Yuanqi Du, Michael S. Albergo arxiv

We study the problem of training diffusion and flow generative models to sample from target distributions defined by an exponential tilting of a base density; a formulation that subsumes both sampling from unnormalized d…