On the capacity of deep generative networks for approximating distributions
We study the efficacy and efficiency of deep generative networks for approximating probability distributions. We prove that neural networks can transform a low-dimensional source distribution to a distribution that is arbitrarily close to a high-dimensional target distribution, when the closeness are measured by Wasserstein distances and maximum mean discrepancy. Upper bounds of the approximation error are obtained in terms of the width and depth of neural network. Furthermore, it is shown that the approximation error in Wasserstein distance grows at most linearly on the ambient dimension and that the approximation order only depends on the intrinsic dimension of the target distribution. On the contrary, when $f$-divergences are used as metrics of distributions, the approximation property is different. We show that in order to approximate the target distribution in $f$-divergences, the dimension of the source distribution cannot be smaller than the intrinsic dimension of the target distribution.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Approximating Probability Distributions by using Wasserstein Generative Adversarial Networks
Studied here are Wasserstein generative adversarial networks (WGANs) with GroupSort neural networks as their discriminators. It is shown that the error bound of the approximation for the target distribution depends on th…
Expanding variational autoencoders for learning and exploiting latent representations in search distributions
In the past, evolutionary algorithms (EAs) that use probabilistic modeling of the best solutions incorporated latent or hidden vari- ables to the models as a more accurate way to represent the search distributions. Recen…
Evolutionary AlgorithmsAdversarial Examples are Misaligned in Diffusion Model Manifolds
In recent years, diffusion models (DMs) have drawn significant attention for their success in approximating data distributions, yielding state-of-the-art generative results. Nevertheless, the versatility of these models …
Adversarial RobustnessImage InpaintingmodelUnderstanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks
Motivated by challenges in conditional generative modeling, where the target conditional density takes the form of a ratio f1 over f2, this paper develops a theoretical framework for approximating such ratio-type functio…
Capacity of Continuous Channels with Memory via Directed Information Neural Estimator
Calculating the capacity (with or without feedback) of channels with memory and continuous alphabets is a challenging task. It requires optimizing the directed information (DI) rate over all channel input distributions. …
Capacity Estimation