Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes
We show how to use unlabeled data and a deep belief net (DBN) to learn a good covariance kernel for a Gaussian process. We first learn a deep generative model of the unlabeled data using the fast, greedy algorithm introduced by Hinton et.al. If the data is high-dimensional and highly-structured, a Gaussian kernel applied to the top layer of features in the DBN works much better than a similar kernel applied to the raw input. Performance at both regression and classification can then be further improved by using backpropagation through the DBN to discriminatively fine-tune the covariance kernel.
Code (0)
등록된 구현이 없습니다.
Tasks
Gaussian ProcessesGeneral ClassificationregressionSimilar Papers 제목 키워드 기반
Random walk kernels and learning curves for Gaussian process regression on random graphs
We consider learning on graphs, guided by kernels that encode similarity between vertices. Our focus is on random walk kernels, the analogues of squared exponential kernels in Euclidean spaces. We show that on large, loc…
Gaussian ProcessesregressionKernel Methods for Deep Learning
We introduce a new family of positive-definite kernel functions that mimic the computation in large, multilayer neural nets. These kernel functions can be used in shallow architectures, such as support vector machines (…
Deep LearningNon-separable Covariance Kernels for Spatiotemporal Gaussian Processes based on a Hybrid Spectral Method and the Harmonic Oscillator
Gaussian processes provide a flexible, non-parametric framework for the approximation of functions in high-dimensional spaces. The covariance kernel is the main engine of Gaussian processes, incorporating correlations th…
Computational EfficiencyGaussian ProcessesPhone Recognition with the Mean-Covariance Restricted Boltzmann Machine
Straightforward application of Deep Belief Nets (DBNs) to acoustic modeling produces a rich distributed representation of speech data that is useful for recognition and yields impressive results on the speaker-independen…
Gaussian Process Kernels for Pattern Discovery and Extrapolation
Gaussian processes are rich distributions over functions, which provide a Bayesian nonparametric approach to smoothing and interpolation. We introduce simple closed form kernels that can be used with Gaussian processes t…
Gaussian Processes