Decoupled Kernel Neural Processes: Neural Network-Parameterized Stochastic Processes using Explicit Data-driven Kernel
Neural Processes (NPs) are a class of stochastic processes parametrized by neural networks. Unlike traditional stochastic processes (e.g., Gaussian processes), which require specifying explicit kernel functions, NPs implicitly learn kernel functions appropriate for a given task through observed data. While this data-driven learning of stochastic processes has been shown to model various types of data, the current NPs' implicit treatment of the mean and the covariance of the output variables limits its full potential when the underlying distribution of the given data is highly complex. To address this, we introduce a new neural stochastic processes, Decoupled Kernel Neural Processes (DKNPs), which explicitly learn a separate mean and kernel function to directly model the covariance between output variables in a data-driven manner. By estimating kernel functions with self- and mixed attentive neural networks, DKNPs demonstrate improved uncertainty estimation in terms of conditional likelihood and diversity in generated samples in 1-D and 2-D regression tasks, compared to other concurrent NP variants. Also, maintaining explicit kernel functions, a key component of stochastic processes, allows the model to reveal a deeper understanding of underlying distributions.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversityGaussian ProcessesSimilar Papers 제목 키워드 기반
An efficient Wasserstein-distance approach for reconstructing jump-diffusion processes using parameterized neural networks
We analyze the Wasserstein distance ($W$-distance) between two probability distributions associated with two multidimensional jump-diffusion processes. Specifically, we analyze a temporally decoupled squared $W_2$-distan…
Decoupled Sparse Gaussian Processes Components]{Decoupled Sparse Gaussian Processes Components : Separating Decision Making from Data Manifold Fitting
We propose a decoupling in Reproducing Kernel Hilbert Space of the parametric and non-parametric components of Sparse Gaussian Processes. We demonstrate that this decoupling results in a set of inducing points concentra…
Decision MakingGaussian ProcessesregressionSignature moments to characterize laws of stochastic processes
The sequence of moments of a vector-valued random variable can characterize its law. We study the analogous problem for path-valued random variables, that is stochastic processes, by using so-called robust signature mome…
Stochastic Gradient Descent for Two-layer Neural Networks
This paper presents a comprehensive study on the convergence rates of the stochastic gradient descent (SGD) algorithm when applied to overparameterized two-layer neural networks. Our approach combines the Neural Tangent …
Learning theory estimates with observations from general stationary stochastic processes
This paper investigates the supervised learning problem with observations drawn from certain general stationary stochastic processes. Here by \emph{general}, we mean that many stationary stochastic processes can be inclu…
Learning Theoryquantile regression