Variational Sequential Optimal Experimental Design using Reinforcement Learning

2023-06-17 · Wanggang Shen, Jiayuan Dong, Xun Huan

We present variational sequential optimal experimental design (vsOED), a novel method for optimally designing a finite sequence of experiments within a Bayesian framework with information-theoretic criteria. vsOED employs a one-point reward formulation with variational posterior approximations, providing a provable lower bound to the expected information gain. Numerical methods are developed following an actor-critic reinforcement learning approach, including derivation and estimation of variational and policy gradients to optimize the design policy, and posterior approximation using Gaussian mixture models and normalizing flows. vsOED accommodates nuisance parameters, implicit likelihoods, and multiple candidate models, while supporting flexible design criteria that can target designs for model discrimination, parameter inference, goal-oriented prediction, and their weighted combinations. We demonstrate vsOED across various engineering and science applications, illustrating its superior sample efficiency compared to existing sequential experimental design algorithms.

📄 PDF Abstract BibTeX arXiv:2306.10430

Code (0)

등록된 구현이 없습니다.

Tasks

Experimental Designreinforcement-learning Reinforcement Learning

SVQN: Sequential Variational Soft Q-Learning Networks

2020-01-01 · ICLR 2020 1 · Shiyu Huang, Hang Su, Jun Zhu, Ting Chen

Partially Observable Markov Decision Processes (POMDPs) are popular and flexible models for real-world decision-making applications that demand the information from past observations to make optimal decisions. Standard r…

Decision Making Q-Learning reinforcement-learning Reinforcement Learning+1

VariBASed: Variational Bayes-Adaptive Sequential Monte-Carlo Planning for Deep Reinforcement Learning

2026-02-21 · Joery A. de Vries, Jinke He, Yaniv Oren, Pascal R. van der Vaart 외 arxiv

Optimally trading-off exploration and exploitation is the holy grail of reinforcement learning as it promises maximal data-efficiency for solving any task. Bayes-optimal agents achieve this, but obtaining the belief-stat…

Reinforcement Learning

Adaptive sampling using variational autoencoder and reinforcement learning

2025-12-03 · Adil Rasheed, Mikael Aleksander Jansen Shahly, Muhammad Faisal Aftab arxiv

Compressed sensing enables sparse sampling but relies on generic bases and random measurements, limiting efficiency and reconstruction quality. Optimal sensor placement uses historcal data to design tailored sampling pat…

Reinforcement Learning

Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learning

2021-10-28 · Wanggang Shen, Xun Huan

We present a mathematical framework and computational methods to optimally design a finite number of sequential experiments. We formulate this sequential optimal experimental design (sOED) problem as a finite-horizon par…

Experimental Design reinforcement-learning Reinforcement Learning (RL)

Gradient-Free Sequential Bayesian Experimental Design via Interacting Particle Systems

2025-04-17 · Robert Gruhlke, Matei Hanu, Claudia Schillings, Philipp Wacker

We introduce a gradient-free framework for Bayesian Optimal Experimental Design (BOED) in sequential settings, aimed at complex systems where gradient information is unavailable. Our method combines Ensemble Kalman Inver…

Experimental Design