Kernel Bayesian Inference with Posterior Regularization
We propose a vector-valued regression problem whose solution is equivalent to the reproducing kernel Hilbert space (RKHS) embedding of the Bayesian posterior distribution. This equivalence provides a new understanding of kernel Bayesian inference. Moreover, the optimization problem induces a new regularization for the posterior embedding estimator, which is faster and has comparable performance to the squared regularization in kernel Bayes' rule. This regularization coincides with a former thresholding approach used in kernel POMDPs whose consistency remains to be established. Our theoretical work solves this open problem and provides consistency analysis in regression settings. Based on our optimizational formulation, we propose a flexible Bayesian posterior regularization framework which for the first time enables us to put regularization at the distribution level. We apply this method to nonparametric state-space filtering tasks with extremely nonlinear dynamics and show performance gains over all other baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceregressionSimilar Papers 제목 키워드 기반
Posterior Consistency of the Silverman g-prior in Bayesian Model Choice
Kernel supervised learning methods can be unified by utilizing the tools from regularization theory. The duality between regularization and prior leads to interpreting regularization methods in terms of maximum a posteri…
Scale invariant process regression: Towards Bayesian ML with minimal assumptions
Current methods for regularization in machine learning require quite specific model assumptions (e.g. a kernel shape) that are not derived from prior knowledge about the application, but must be imposed merely to make th…
Bayesian InferenceGaussian ProcessesregressionTranslationBayesian Deconditional Kernel Mean Embeddings
Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the or…
Gaussian ProcessesEfficient Approximate Inference with Walsh-Hadamard Variational Inference
Variational inference offers scalable and flexible tools to tackle intractable Bayesian inference of modern statistical models like Bayesian neural networks and Gaussian processes. For largely over-parameterized models, …
Bayesian InferenceGaussian ProcessesVariational InferencePosterior Regularization on Bayesian Hierarchical Mixture Clustering
Bayesian hierarchical mixture clustering (BHMC) improves traditionalBayesian hierarchical clustering by replacing conventional Gaussian-to-Gaussian kernels with a Hierarchical Dirichlet Process Mixture Model(HDPMM) for p…
Clustering