Precision annealing Monte Carlo methods for statistical data assimilation and machine learning
In statistical data assimilation (SDA) and supervised machine learning (ML), we wish to transfer information from observations to a model of the processes underlying those observations. For SDA, the model consists of a set of differential equations that describe the dynamics of a physical system. For ML, the model is usually constructed using other strategies. In this paper, we develop a systematic formulation based on Monte Carlo sampling to achieve such information transfer. Following the derivation of an appropriate target distribution, we present the formulation based on the standard Metropolis-Hasting (MH) procedure and the Hamiltonian Monte Carlo (HMC) method for performing the high dimensional integrals that appear. To the extensive literature on MH and HMC, we add (1) an annealing method using a hyperparameter that governs the precision of the model to identify and explore the highest probability regions of phase space dominating those integrals, and (2) a strategy for initializing the state space search. The efficacy of the proposed formulation is demonstrated using a nonlinear dynamical model with chaotic solutions widely used in geophysics.
Code (0)
등록된 구현이 없습니다.
Tasks
BIG-bench Machine LearningGeophysicsSimilar Papers 제목 키워드 기반
Toward Unlimited Self-Learning MCMC with Parallel Adaptive Annealing
Self-learning Monte Carlo (SLMC) methods are recently proposed to accelerate Markov chain Monte Carlo (MCMC) methods using a machine learning model. With latent generative models, SLMC methods realize efficient Monte Car…
Self-LearningBayesian multiple change-point estimation with annealing stochastic approximation Monte Carlo
Bayesian multiple change-point models are built with data from normal, exponential, binomial and Poisson distributions with a truncated Poisson prior for the number of change-points and conjugate prior for the distributi…
Ergodic Annealing
Simulated Annealing is the crowning glory of Markov Chain Monte Carlo Methods for the solution of NP-hard optimization problems in which the cost function is known. Here, by replacing the Metropolis engine of Simulated A…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)An Annealed Sequential Monte Carlo Method for Bayesian Phylogenetics
We describe an "embarrassingly parallel" method for Bayesian phylogenetic inference, annealed Sequential Monte Carlo, based on recent advances in the Sequential Monte Carlo literature such as adaptive determination of an…
Demonstrating Real Advantage of Machine-Learning-Enhanced Monte Carlo for Combinatorial Optimization
Combinatorial optimization problems are central to both practical applications and the development of optimization methods. While classical and quantum algorithms have been refined over decades, machine learning--assiste…