A Deep Learning Method for Comparing Bayesian Hierarchical Models
Bayesian model comparison (BMC) offers a principled approach for assessing the relative merits of competing computational models and propagating uncertainty into model selection decisions. However, BMC is often intractable for the popular class of hierarchical models due to their high-dimensional nested parameter structure. To address this intractability, we propose a deep learning method for performing BMC on any set of hierarchical models which can be instantiated as probabilistic programs. Since our method enables amortized inference, it allows efficient re-estimation of posterior model probabilities and fast performance validation prior to any real-data application. In a series of extensive validation studies, we benchmark the performance of our method against the state-of-the-art bridge sampling method and demonstrate excellent amortized inference across all BMC settings. We then showcase our method by comparing four hierarchical evidence accumulation models that have previously been deemed intractable for BMC due to partly implicit likelihoods. Additionally, we demonstrate how transfer learning can be leveraged to enhance training efficiency. We provide reproducible code for all analyses and an open-source implementation of our method.
Code (3)
Tasks
Decision MakingDeep LearningModel SelectionTransfer LearningSimilar Papers 제목 키워드 기반
Target Identification and Bayesian Model Averaging with Probabilistic Hierarchical Factor Probabilities
Target detection in hyperspectral imagery is the process of locating pixels from an image which are likely to contain target, typically done by comparing one or more spectra for the desired target material to each pixel …
Bayesian Trading Cost Analysis and Ranking of Broker Algorithms
We present a formulation of the transaction cost analysis (TCA) in the Bayesian framework for the primary purpose of comparing broker algorithms using standardized benchmarks. Our formulation allows effective calculation…
Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation
Gaussian process regression has proven very powerful in statistics, machine learning and inverse problems. A crucial aspect of the success of this methodology, in a wide range of applications to complex and real-world pr…
BIG-bench Machine Learningparameter estimationBayesian inference of infected patients in group testing with prevalence estimation
Group testing is a method of identifying infected patients by performing tests on a pool of specimens collected from patients. For the case in which the test returns a false result with finite probability, we propose Bay…
Bayesian InferenceShort Text Language Identification for Under Resourced Languages
The paper presents a hierarchical naive Bayesian and lexicon based classifier for short text language identification (LID) useful for under resourced languages. The algorithm is evaluated on short pieces of text for the …
Language Identification