Adaptive Cholesky Gaussian Processes
We present a method to approximate Gaussian process regression models for large datasets by considering only a subset of the data. Our approach is novel in that the size of the subset is selected on the fly during exact inference with little computational overhead. From an empirical observation that the log-marginal likelihood often exhibits a linear trend once a sufficient subset of a dataset has been observed, we conclude that many large datasets contain redundant information that only slightly affects the posterior. Based on this, we provide probabilistic bounds on the full model evidence that can identify such subsets. Remarkably, these bounds are largely composed of terms that appear in intermediate steps of the standard Cholesky decomposition, allowing us to modify the algorithm to adaptively stop the decomposition once enough data have been observed.
Code (2)
Tasks
Gaussian ProcessesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Variational sparse inverse Cholesky approximation for latent Gaussian processes via double Kullback-Leibler minimization
To achieve scalable and accurate inference for latent Gaussian processes, we propose a variational approximation based on a family of Gaussian distributions whose covariance matrices have sparse inverse Cholesky (SIC) fa…
Gaussian ProcessesSparse Cholesky Factorization for Solving Nonlinear PDEs via Gaussian Processes
In recent years, there has been widespread adoption of machine learning-based approaches to automate the solving of partial differential equations (PDEs). Among these approaches, Gaussian processes (GPs) and kernel metho…
Gaussian ProcessesThe Geometry of the Pivot: A Note on Lazy Pivoted Cholesky and Farthest Point Sampling
Low-rank approximations of large kernel matrices are ubiquitous in machine learning, particularly for scaling Gaussian Processes to massive datasets. The Pivoted Cholesky decomposition is a standard tool for this task, o…
Gaussian ProcessesScalable Grouped Gaussian Processes via Direct Cholesky Functional Representations
We consider multi-task regression models where observations are assumed to be a linear combination of several latent node and weight functions, all drawn from Gaussian process (GP) priors that allow nonzero covariance be…
Gaussian ProcessesVariational InferenceNovel Pivoted Cholesky Decompositions for Efficient Gaussian Process Inference
The Cholesky decomposition is a fundamental tool for solving linear systems with symmetric and positive definite matrices which are ubiquitous in linear algebra, optimization, and machine learning. Its numerical stabilit…
Gaussian ProcessesActive Learning