Optimal Neural Network Approximation of Wasserstein Gradient Direction via Convex Optimization
The computation of Wasserstein gradient direction is essential for posterior sampling problems and scientific computing. The approximation of the Wasserstein gradient with finite samples requires solving a variational problem. We study the variational problem in the family of two-layer networks with squared-ReLU activations, towards which we derive a semi-definite programming (SDP) relaxation. This SDP can be viewed as an approximation of the Wasserstein gradient in a broader function family including two-layer networks. By solving the convex SDP, we obtain the optimal approximation of the Wasserstein gradient direction in this class of functions. Numerical experiments including PDE-constrained Bayesian inference and parameter estimation in COVID-19 modeling demonstrate the effectiveness of the proposed method.
Code (1)
Tasks
Bayesian Inferenceparameter estimationSimilar Papers 제목 키워드 기반
Hessian-guided Perturbed Wasserstein Gradient Flows for Escaping Saddle Points
Wasserstein gradient flow (WGF) is a common method to perform optimization over the space of probability measures. While WGF is guaranteed to converge to a first-order stationary point, for nonconvex functionals the conv…
Stochastic Approximation versus Sample Average Approximation for population Wasserstein barycenters
In the machine learning and optimization community, there are two main approaches for the convex risk minimization problem, namely, the Stochastic Approximation (SA) and the Sample Average Approximation (SAA). In terms o…
2-Wasserstein Approximation via Restricted Convex Potentials with Application to Improved Training for GANs
We provide a framework to approximate the 2-Wasserstein distance and the optimal transport map, amenable to efficient training as well as statistical and geometric analysis. With the quadratic cost and considering the Ka…
From Saddle Points Toward Global Minima: A Newton-Type Method on Wasserstein Space
We study the minimization of non-convex functionals over the Wasserstein space. While recent work has showed that perturbed Wasserstein gradient methods can avoid saddle points for benign landscapes, existing approaches …
Particle-based Variational Inference with Generalized Wasserstein Gradient Flow
Particle-based variational inference methods (ParVIs) such as Stein variational gradient descent (SVGD) update the particles based on the kernelized Wasserstein gradient flow for the Kullback-Leibler (KL) divergence. How…