Predictively Oriented Posteriors
We advocate for a new statistical principle that combines the most desirable aspects of both parameter inference and density estimation. This leads us to the predictively oriented (PrO) posterior, which expresses uncertainty as a consequence of predictive ability. We show that these posteriors converge to the predictively optimal model average and predictively dominate both classical and generalised Bayes posterior predictive distributions. Further, PrO posteriors adapt to the level of model misspecification: while they concentrate around the true model in the same way as classical and generalised Bayesian strategies if the model can recover the data-generating distribution, they do not concentrate around a single model in the presence of non-trivial forms of model misspecification. Instead, they stabilise towards a non-degenerate predictively optimal posterior distribution that represents a form of irreducible uncertainty due to model misspecification. We put forward a sampling algorithm for PrO posteriors based on mean field Langevin dynamics, and verify the practical significance of our theoretical developments on a number of numerical examples.
Code (0)
등록된 구현이 없습니다.
Tasks
Density EstimationSimilar Papers 제목 키워드 기반
Thinned Mean Field Langevin Dynamics
Several important learning tasks can be formulated as minimizing an entropy-regularized objective over an appropriate space of probability distributions. Mean-field Langevin dynamics (MFLD) facilitate computation in this…
A Computable Measure of Suboptimality for Entropy-Regularised Variational Objectives
Several methods in statistics and machine learning target a probability distribution for which an entropy-regularised variational objective is minimised. This increased flexibility introduces a computational challenge, a…
Taming Continuous Posteriors for Latent Variational Dialogue Policies
Utilizing amortized variational inference for latent-action reinforcement learning (RL) has been shown to be an effective approach in Task-oriented Dialogue (ToD) systems for optimizing dialogue success. Until now, categ…
reinforcement-learningReinforcement Learning (RL)Variational InferenceMultiple Predictively Equivalent Risk Models for Handling Missing Data at Time of Prediction: with an Application in Severe Hypoglycemia Risk Prediction for Type 2 Diabetes
The presence of missing data at the time of prediction limits the application of risk models in clinical and research settings. Common ways of handling missing data at the time of prediction include measuring the missi…
ImputationMissing ValuesPredictionPredictively Combatting Toxicity in Health-related Online Discussions through Machine Learning
In health-related topics, user toxicity in online discussions frequently becomes a source of social conflict or promotion of dangerous, unscientific behaviour; common approaches for battling it include different forms of…
Collaborative Filtering