When are Iterative Gaussian Processes Reliably Accurate?
While recent work on conjugate gradient methods and Lanczos decompositions have achieved scalable Gaussian process inference with highly accurate point predictions, in several implementations these iterative methods appear to struggle with numerical instabilities in learning kernel hyperparameters, and poor test likelihoods. By investigating CG tolerance, preconditioner rank, and Lanczos decomposition rank, we provide a particularly simple prescription to correct these issues: we recommend that one should use a small CG tolerance ($\epsilon \leq 0.01$) and a large root decomposition size ($r \geq 5000$). Moreover, we show that L-BFGS-B is a compelling optimizer for Iterative GPs, achieving convergence with fewer gradient updates.
Code (1)
Tasks
Gaussian ProcessesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Warm-Starting Iterative Gaussian Processes for Faster Sequential Inference
Efficient Gaussian process (GP) inference is critical for sequential decision-making tasks such as active learning, online prediction, and Bayesian optimization. Iterative approaches of approximating the GP posterior usi…
Gaussian ProcessesActive LearningDistributional Gaussian Processes Layers for Out-of-Distribution Detection
Machine learning models deployed on medical imaging tasks must be equipped with out-of-distribution detection capabilities in order to avoid erroneous predictions. It is unsure whether out-of-distribution detection model…
Gaussian ProcessesOut-of-Distribution DetectionSegmentationThe Elliptical Processes: a Family of Fat-tailed Stochastic Processes
We present the elliptical processes -- a family of non-parametric probabilistic models that subsumes the Gaussian process and the Student-t process. This generalization includes a range of new fat-tailed behaviors yet re…
Gaussian ProcessesregressionDistributional Gaussian Process Layers for Outlier Detection in Image Segmentation
We propose a parameter efficient Bayesian layer for hierarchical convolutional Gaussian Processes that incorporates Gaussian Processes operating in Wasserstein-2 space to reliably propagate uncertainty. This directly rep…
Gaussian ProcessesImage SegmentationOutlier DetectionOut-of-Distribution Detection+2Variational Elliptical Processes
We present elliptical processes, a family of non-parametric probabilistic models that subsume Gaussian processes and Student's t processes. This generalization includes a range of new heavy-tailed behaviors while retaini…
Gaussian ProcessesVariational Inference