Score Operator Newton transport
We propose a new approach for sampling and Bayesian computation that uses the score of the target distribution to construct a transport from a given reference distribution to the target. Our approach is an infinite-dimensional Newton method, involving a linear PDE, for finding a zero of a ``score-residual'' operator. We prove sufficient conditions for convergence to a valid transport map. Our Newton iterates can be computed by exploiting fast solvers for elliptic PDEs, resulting in new algorithms for Bayesian inference and other sampling tasks. We identify elementary settings where score-operator Newton transport achieves fast convergence while avoiding mode collapse.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferencevalidVariational InferenceMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Riemannian stochastic quasi-Newton algorithm with variance reduction and its convergence analysis
Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large, but finite number of loss functions. The present paper proposes a Riemannian stochastic quasi-Newton algorithm …
Low-Rank Matrix CompletionMatrix CompletionSolving Newton's Equations of Motion with Large Timesteps using Recurrent Neural Networks based Operators
Classical molecular dynamics simulations are based on solving Newton's equations of motion. Using a small timestep, numerical integrators such as Verlet generate trajectories of particles as solutions to Newton's equatio…
A Neural-Operator Preconditioned Newton Method for Accelerated Nonlinear Solvers
We propose a novel neural preconditioned Newton (NP-Newton) method for solving parametric nonlinear systems of equations. To overcome the stagnation or instability of Newton iterations caused by unbalanced nonlinearities…
Computational EfficiencyNeural-Initialized Newton: Accelerating Nonlinear Finite Elements via Operator Learning
We propose a Newton-based scheme, initialized by neural operator predictions, to accelerate the parametric solution of nonlinear problems in computational solid mechanics. First, a physics informed conditional neural fie…
Differentiable Top-k Operator with Optimal Transport
The top-k operation, i.e., finding the k largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining. Howeve…
Information RetrievalRetrieval