Scalable Lévy Process Priors for Spectral Kernel Learning
Gaussian processes are rich distributions over functions, with generalization properties determined by a kernel function. When used for long-range extrapolation, predictions are particularly sensitive to the choice of kernel parameters. It is therefore critical to account for kernel uncertainty in our predictive distributions. We propose a distribution over kernels formed by modelling a spectral mixture density with a L\'evy process. The resulting distribution has support for all stationary covariances--including the popular RBF, periodic, and Mat\'ern kernels--combined with inductive biases which enable automatic and data efficient learning, long-range extrapolation, and state of the art predictive performance. The proposed model also presents an approach to spectral regularization, as the L\'evy process introduces a sparsity-inducing prior over mixture components, allowing automatic selection over model order and pruning of extraneous components. We exploit the algebraic structure of the proposed process for $\mathcal{O}(n)$ training and $\mathcal{O}(1)$ predictions. We perform extrapolations having reasonable uncertainty estimates on several benchmarks, show that the proposed model can recover flexible ground truth covariances and that it is robust to errors in initialization.
Code (1)
Tasks
Gaussian ProcessesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Scalable Levy Process Priors for Spectral Kernel Learning
Gaussian processes are rich distributions over functions, with generalization properties determined by a kernel function. When used for long-range extrapolation, predictions are particularly sensitive to the choice of ke…
Gaussian ProcessesNeural Non-Stationary Spectral Kernel
Standard kernels such as Mat\'ern or RBF kernels only encode simple monotonic dependencies within the input space. Spectral mixture kernels have been proposed as general-purpose, flexible kernels for learning and discove…
Gaussian ProcessesDeep Kernel Learning
We introduce scalable deep kernels, which combine the structural properties of deep learning architectures with the non-parametric flexibility of kernel methods. Specifically, we transform the inputs of a spectral mixtur…
Gaussian ProcessesSpectral Mixture Kernel Approximation Using Reparameterized Random Fourier Feature
We propose a method for Spectral Mixture kernel approximation using the Reparameterized Random Fourier Feature (R-RFF) in the sense of both general parameter and natural parameter view. Meanwhile, we provide the effectiv…
Hida-Matérn Kernel
We present the class of Hida-Mat\'ern kernels, which is the canonical family of covariance functions over the entire space of stationary Gauss-Markov Processes. It extends upon Mat\'ern kernels, by allowing for flexible …
State Space Models