Non-asymptotic approximations of neural networks by Gaussian processes
We study the extent to which wide neural networks may be approximated by Gaussian processes when initialized with random weights. It is a well-established fact that as the width of a network goes to infinity, its law converges to that of a Gaussian process. We make this quantitative by establishing explicit convergence rates for the central limit theorem in an infinite-dimensional functional space, metrized with a natural transportation distance. We identify two regimes of interest; when the activation function is polynomial, its degree determines the rate of convergence, while for non-polynomial activations, the rate is governed by the smoothness of the function.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesSimilar Papers 제목 키워드 기반
Non-asymptotic approximations of Gaussian neural networks via second-order Poincaré inequalities
There is a growing interest on large-width asymptotic properties of Gaussian neural networks (NNs), namely NNs whose weights are initialized according to Gaussian distributions. A well-established result is that, as the …
Multiscale Sparse Microcanonical Models
We study approximations of non-Gaussian stationary processes having long range correlations with microcanonical models. These models are conditioned by the empirical value of an energy vector, evaluated on a single reali…
Gaussian ProcessesPoint ProcessesTexture SynthesisStudent-t processes as infinite-width limits of posterior Bayesian neural networks
The asymptotic properties of Bayesian Neural Networks (BNNs) have been extensively studied, particularly regarding their approximations by Gaussian processes in the infinite-width limit. We extend these results by showin…
Gaussian ProcessesVolatility options in rough volatility models
We discuss the pricing and hedging of volatility options in some rough volatility models. First, we develop efficient Monte Carlo methods and asymptotic approximations for computing option prices and hedge ratios in mode…
GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration
Despite advances in scalable models, the inference tools used for Gaussian processes (GPs) have yet to fully capitalize on developments in computing hardware. We present an efficient and general approach to GP inference …
Gaussian ProcessesGPU