Bayesian Properties of Normalized Maximum Likelihood and its Fast Computation
The normalized maximized likelihood (NML) provides the minimax regret solution in universal data compression, gambling, and prediction, and it plays an essential role in the minimum description length (MDL) method of statistical modeling and estimation. Here we show that the normalized maximum likelihood has a Bayes-like representation as a mixture of the component models, even in finite samples, though the weights of linear combination may be both positive and negative. This representation addresses in part the relationship between MDL and Bayes modeling. This representation has the advantage of speeding the calculation of marginals and conditionals required for coding and prediction applications.
Code (0)
등록된 구현이 없습니다.
Tasks
Data CompressionPredictionSimilar Papers 제목 키워드 기반
Quotient Normalized Maximum Likelihood Criterion for Learning Bayesian Network Structures
We introduce an information theoretic criterion for Bayesian network structure learning which we call quotient normalized maximum likelihood (qNML). In contrast to the closely related factorized normalized maximum likeli…
Amortized Conditional Normalized Maximum Likelihood
While deep neural networks provide good performance for a range of challenging tasks, calibration and uncertainty estimation remain major challenges. In this paper, we propose the amortized conditional normalized maximum…
Bayesian InferenceAmortized Conditional Normalized Maximum Likelihood: Reliable Out of Distribution Uncertainty Estimation
While deep neural networks provide good performance for a range of challenging tasks, calibration and uncertainty estimation remain major challenges, especially under distribution shift. In this paper, we propose the amo…
Bayesian InferenceRenormalized Normalized Maximum Likelihood and Three-Part Code Criteria For Learning Gaussian Networks
Score based learning (SBL) is a promising approach for learning Bayesian networks in the discrete domain. However, when employing SBL in the continuous domain, one is either forced to move the problem to the discrete dom…
MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy
In some misspecified settings, the posterior distribution in Bayesian statistics may lead to inconsistent estimates. To fix this issue, it has been suggested to replace the likelihood by a pseudo-likelihood, that is the …