Tensor Network-Constrained Kernel Machines as Gaussian Processes
Tensor Networks (TNs) have recently been used to speed up kernel machines by constraining the model weights, yielding exponential computational and storage savings. In this paper we prove that the outputs of Canonical Polyadic Decomposition (CPD) and Tensor Train (TT)-constrained kernel machines recover a Gaussian Process (GP), which we fully characterize, when placing i.i.d. priors over their parameters. We analyze the convergence of both CPD and TT-constrained models, and show how TT yields models exhibiting more GP behavior compared to CPD, for the same number of model parameters. We empirically observe this behavior in two numerical experiments where we respectively analyze the convergence to the GP and the performance at prediction. We thereby establish a connection between TN-constrained kernel machines and GPs.
Code (1)
Tasks
Gaussian ProcessesTensor NetworksMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Laplace Approximation for Bayesian Tensor Network Kernel Machines
Uncertainty estimation is essential for robust decision-making in the presence of ambiguous or out-of-distribution inputs. Gaussian Processes (GPs) are classical kernel-based models that offer principled uncertainty quan…
Bayesian InferenceGaussian ProcessesTensor Regression Meets Gaussian Processes
Low-rank tensor regression, a new model class that learns high-order correlation from data, has recently received considerable attention. At the same time, Gaussian processes (GP) are well-studied machine learning models…
Bayesian InferenceGaussian ProcessesregressionScalable Gaussian Processes on Discrete Domains
Kernel methods on discrete domains have shown great promise for many challenging data types, for instance, biological sequence data and molecular structure data. Scalable kernel methods like Support Vector Machines may o…
Gaussian ProcessesPoint ProcessesAdapting Tensor Kernel Machines to Enable Efficient Transfer Learning for Seizure Detection
Transfer learning aims to optimize performance in a target task by learning from a related source problem. In this work, we propose an efficient transfer learning method using a tensor kernel machine. Our method takes in…
Transfer LearningSeizure DetectionLarge Scale Tensor Regression using Kernels and Variational Inference
We outline an inherent weakness of tensor factorization models when latent factors are expressed as a function of side information and propose a novel method to mitigate this weakness. We coin our method \textit{Kernel F…
regressionVariational Inference