Bayesian Context Aggregation for Neural Processes
Formulating scalable probabilistic regression models with reliable uncertainty estimates has been a long-standing challenge in machine learning research. Recently, casting probabilistic regression as a multi-task learning problem in terms of conditional latent variable (CLV) models such as the Neural Process (NP) has shown promising results. In this paper, we focus on context aggregation, a central component of such architectures, which fuses information from multiple context data points. So far, this aggregation operation has been treated separately from the inference of a latent representation of the target function in CLV models. Our key contribution is to combine these steps into one holistic mechanism by phrasing context aggregation as a Bayesian inference problem. The resulting Bayesian Aggregation (BA) mechanism allows principled handling of task ambiguity, which is key for efficiently processing context information. We demonstrate on a range of challenging experiments, including dynamics modeling and image completion, that BA consistently improves upon the performance of traditional aggregation methods while remaining computationally efficient and fully compatible with existing NP-based models.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceMulti-Task LearningregressionSimilar Papers 제목 키워드 기반
Graph Neural Processes for Spatio-Temporal Extrapolation
We study the task of spatio-temporal extrapolation that generates data at target locations from surrounding contexts in a graph. This task is crucial as sensors that collect data are sparsely deployed, resulting in a lac…
Gaussian Processesspatio-temporal extrapolationGeometry-Aware Hierarchical Bayesian Learning on Manifolds
Bayesian learning with Gaussian processes demonstrates encouraging regression and classification performances in solving computer vision tasks. However, Bayesian methods on 3D manifold-valued vision data, such as meshes …
Gaussian ProcessesregressionNonparametric Bayesian Topic Modelling with the Hierarchical Pitman-Yor Processes
The Dirichlet process and its extension, the Pitman-Yor process, are stochastic processes that take probability distributions as a parameter. These processes can be stacked up to form a hierarchical nonparametric Bayesia…
Topic ModelsBayesian Meta-Learning Through Variational Gaussian Processes
Recent advances in the field of meta-learning have tackled domains consisting of large numbers of small ("few-shot") supervised learning tasks. Meta-learning algorithms must be able to rapidly adapt to any individual few…
Gaussian ProcessesMeta-LearningFusion of Gaussian Processes Predictions with Monte Carlo Sampling
In science and engineering, we often work with models designed for accurate prediction of variables of interest. Recognizing that these models are approximations of reality, it becomes desirable to apply multiple models …
Gaussian Processes