Continuous-Time Birth-Death MCMC for Bayesian Regression Tree Models
Decision trees are flexible models that are well suited for many statistical regression problems. In a Bayesian framework for regression trees, Markov Chain Monte Carlo (MCMC) search algorithms are required to generate samples of tree models according to their posterior probabilities. The critical component of such an MCMC algorithm is to construct good Metropolis-Hastings steps for updating the tree topology. However, such algorithms frequently suffering from local mode stickiness and poor mixing. As a result, the algorithms are slow to converge. Hitherto, authors have primarily used discrete-time birth/death mechanisms for Bayesian (sums of) regression tree models to explore the model space. These algorithms are efficient only if the acceptance rate is high which is not always the case. Here we overcome this issue by developing a new search algorithm which is based on a continuous-time birth-death Markov process. This search algorithm explores the model space by jumping between parameter spaces corresponding to different tree structures. In the proposed algorithm, the moves between models are always accepted which can dramatically improve the convergence and mixing properties of the MCMC algorithm. We provide theoretical support of the algorithm for Bayesian regression tree models and demonstrate its performance.
Code (1)
Tasks
regressionSimilar Papers 제목 키워드 기반
The scalable Birth-Death MCMC Algorithm for Mixed Graphical Model Learning with Application to Genomic Data Integration
Recent advances in biological research have seen the emergence of high-throughput technologies with numerous applications that allow the study of biological mechanisms at an unprecedented depth and scale. A large amount …
Computational EfficiencyData IntegrationModel SelectionBayesian tracking and parameter learning for non-linear multiple target tracking models
We propose a new Bayesian tracking and parameter learning algorithm for non-linear non-Gaussian multiple target tracking (MTT) models. We design a Markov chain Monte Carlo (MCMC) algorithm to sample from the posterior di…
Joint modeling of multiple time series via the beta process with application to motion capture segmentation
We propose a Bayesian nonparametric approach to the problem of jointly modeling multiple related time series. Our model discovers a latent set of dynamical behaviors shared among the sequences, and segments each time ser…
Time SeriesTime Series AnalysisTransforming Worlds: Automated Involutive MCMC for Open-Universe Probabilistic Models
Open-universe probabilistic models enable Bayesian inference about how many objects underlie data, and how they are related. Effective inference in OUPMs remains a challenge, however, often requiring the use of custom, t…
Bayesian InferenceProbabilistic ProgrammingvalidBayesian Nonparametric View to Spawning
In tracking multiple objects, it is often assumed that each observation (measurement) is originated from one and only one object. However, we may encounter a situation that each measurement may or may not be associated w…