Universality of Bayesian mixture predictors
The problem is that of sequential probability forecasting for finite-valued time series. The data is generated by an unknown probability distribution over the space of all one-way infinite sequences. It is known that this measure belongs to a given set C, but the latter is completely arbitrary (uncountably infinite, without any structure given). The performance is measured with asymptotic average log loss. In this work it is shown that the minimax asymptotic performance is always attainable, and it is attained by a convex combination of a countably many measures from the set C (a Bayesian mixture). This was previously only known for the case when the best achievable asymptotic error is 0. This also contrasts previous results that show that in the non-realizable case all Bayesian mixtures may be suboptimal, while there is a predictor that achieves the optimal performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Time SeriesTime Series AnalysisSimilar Papers 제목 키워드 기반
Universal time-series forecasting with mixture predictors
This book is devoted to the problem of sequential probability forecasting, that is, predicting the probabilities of the next outcome of a growing sequence of observations given the past. This problem is considered in a v…
Time SeriesTime Series AnalysisTime Series ForecastingBayesian shrinkage in mixture of experts models: Identifying robust determinants of class membership
A method for implicit variable selection in mixture of experts frameworks is proposed. We introduce a prior structure where information is taken from a set of independent covariates. Robust class membership predictors ar…
Bayesian InferenceMixture-of-ExpertsVariable SelectionThe Breakdown of Gaussian Universality in Classification of High-dimensional Linear Factor Mixtures
The assumption of Gaussian or Gaussian mixture data has been extensively exploited in a long series of precise performance analyses of machine learning (ML) methods, on large datasets having comparably numerous samples a…
Covariate Dependent Mixture of Bayesian Networks
Learning the structure of Bayesian networks from data provides insights into underlying processes and the causal relationships that generate the data, but its usefulness depends on the homogeneity of the data population,…
Depth induces scale-averaging in overparameterized linear Bayesian neural networks
Inference in deep Bayesian neural networks is only fully understood in the infinite-width limit, where the posterior flexibility afforded by increased depth washes out and the posterior predictive collapses to a shallow …
Representation Learning