Sampling Decisions
In this manuscript we introduce a novel Decision Flow (DF) framework for sampling from a target distribution while incorporating additional guidance from a prior sampler. DF can be viewed as an AI driven algorithmic reincarnation of the Markov Decision Process (MDP) approach in Stochastic Optimal Control. It extends the continuous space, continuous time path Integral Diffusion sampling technique to discrete time and space, while also generalizing the Generative Flow Network framework. In its most basic form, an explicit, Neural Network (NN) free formulation, DF leverages the linear solvability of the the underlying MDP to adjust the transition probabilities of the prior sampler. The resulting Markov Process is expressed as a convolution of the reverse time Green's function of the prior sampling with the target distribution. We illustrate the DF framework through an example of sampling from the Ising model, discuss potential NN based extensions, and outline how DF can enhance guided sampling across various applications.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Active Sampling for MRI-based Sequential Decision Making
Despite the superior diagnostic capability of Magnetic Resonance Imaging (MRI), its use as a Point-of-Care (PoC) device remains limited by high cost and complexity. To enable such a future by reducing the magnetic field …
Decision MakingDiagnosticMulti-Objective Reinforcement LearningSequential Decision Making+1Contextual Thompson Sampling via Generation of Missing Data
We introduce a framework for Thompson sampling contextual bandit algorithms, in which the algorithm's ability to quantify uncertainty and make decisions depends on the quality of a generative model that is learned offlin…
Decision MakingFairnessThompson SamplingStatistical Inference After Adaptive Sampling for Longitudinal Data
Online reinforcement learning and other adaptive sampling algorithms are increasingly used in digital intervention experiments to optimize treatment delivery for users over time. In this work, we focus on longitudinal us…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Decision-Making with Auto-Encoding Variational Bayes
To make decisions based on a model fit with auto-encoding variational Bayes (AEVB), practitioners often let the variational distribution serve as a surrogate for the posterior distribution. This approach yields biased es…
Decision MakingTwo-sample testingStatistical inference in social networks: how sampling bias and uncertainty shape decisions
We investigate how individuals form expectations about population behavior using statistical inference based on observations of their social relations. Misperceptions about others' connectedness and behavior arise from s…