Locking and Quacking: Stacking Bayesian model predictions by log-pooling and superposition
Combining predictions from different models is a central problem in Bayesian inference and machine learning more broadly. Currently, these predictive distributions are almost exclusively combined using linear mixtures such as Bayesian model averaging, Bayesian stacking, and mixture of experts. Such linear mixtures impose idiosyncrasies that might be undesirable for some applications, such as multi-modality. While there exist alternative strategies (e.g. geometric bridge or superposition), optimising their parameters usually involves computing an intractable normalising constant repeatedly. We present two novel Bayesian model combination tools. These are generalisations of model stacking, but combine posterior densities by log-linear pooling (locking) and quantum superposition (quacking). To optimise model weights while avoiding the burden of normalising constants, we investigate the Hyvarinen score of the combined posterior predictions. We demonstrate locking with an illustrative example and discuss its practical application with importance sampling.
Code (0)
등록된 구현이 없습니다.
Tasks
Bayesian InferenceMixture-of-ExpertsSimilar Papers 제목 키워드 기반
Bayesian hierarchical stacking: Some models are (somewhere) useful
Stacking is a widely used model averaging technique that asymptotically yields optimal predictions among linear averages. We show that stacking is most effective when model predictive performance is heterogeneous in inpu…
Bayesian InferenceTime SeriesTime Series AnalysisBayesian Regression Approach for Building and Stacking Predictive Models in Time Series Analytics
The paper describes the use of Bayesian regression for building time series models and stacking different predictive models for time series. Using Bayesian regression for time series modeling with nonlinear trend was ana…
Decision MakingregressionTime SeriesTime Series Analysis+1BayesBlend: Easy Model Blending using Pseudo-Bayesian Model Averaging, Stacking and Hierarchical Stacking in Python
Averaging predictions from multiple competing inferential models frequently outperforms predictions from any single model, providing that models are optimally weighted to maximize predictive performance. This is particul…
modelAn Analysis on Ensemble Learning optimized Medical Image Classification with Deep Convolutional Neural Networks
Novel and high-performance medical image classification pipelines are heavily utilizing ensemble learning strategies. The idea of ensemble learning is to assemble diverse models or multiple predictions and, thus, boost p…
ClassificationEnsemble LearningImage Augmentationimage-classification+3Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support
The posterior in probabilistic programs with stochastic support decomposes as a weighted sum of the local posterior distributions associated with each possible program path. We show that making predictions with this full…