Nested Policy Reinforcement Learning for Clinical Decision Support

2021-09-29 · Aishwarya Mandyam, Andrew Jones, Krzysztof Laudanski, Barbara Engelhardt

Off-policy reinforcement learning (RL) has proven to be a powerful framework for guiding agents' actions in environments with stochastic rewards and unknown or noisy state dynamics. In many real-world settings, these agents must operate in multiple environments, each with slightly different dynamics. For example, we may be interested in developing policies to guide medical treatment for patients with and without a given disease, or policies to navigate curriculum design for students with and without a learning disability. Here, we introduce nested policy fitted Q-iteration (NFQI), an RL framework that finds optimal policies in environments that exhibit such a structure. Our approach develops a nested $Q$-value function that takes advantage of the shared structure between two groups of observations from two separate environments while allowing their policies to be distinct from one another. We find that NFQI yields policies that rely on relevant features and perform at least as well as a policy that does not consider group structure. We demonstrate NFQI's performance using an OpenAI Gym environment and a clinical decision making RL task. Our results suggest that NFQI can develop policies that are better suited to many real-world clinical environments.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Decision MakingNavigateOpenAI Gym reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

MedDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Support

2025-05-26 · Qianyi Xu, Gousia Habib, Dilruk Perera, Mengling Feng

Timely and personalized treatment decisions are essential across a wide range of healthcare settings where patient responses vary significantly and evolve over time. Clinical data used to support these decisions are ofte…

Imputation Model-based Reinforcement Learning Recommendation Systems Reinforcement Learning (RL)

Reinforcement Learning enhanced Online Adaptive Clinical Decision Support via Digital Twin powered Policy and Treatment Effect optimized Reward

2025-08-24 · Xinyu Qin, Ruiheng Yu, Lu Wang arxiv

Clinical decision support must adapt online under safety constraints. We present an online adaptive tool where reinforcement learning provides the policy, a patient digital twin provides the environment, and treatment ef…

Reinforcement Learning

Offline Reinforcement Learning for Hemodynamic Management of Sepsis in the ICU: a MIMIC-IV Study with Dual Off-Policy Evaluation

2026-08-17 · Marc Pérez-Roig, David Fernández-Narro, Carlos Sáez arxiv

The dosing of intravenous fluids and vasopressors in sepsis is a sequential decision made under uncertainty and guided largely by clinical judgment, which makes it a natural target for reinforcement learning from histori…

Reinforcement Learning

Deep Reinforcement Learning for Clinical Decision Support: A Brief Survey

2019-07-22 · Si-Qi Liu, Kee Yuan Ngiam, Mengling Feng

Owe to the recent advancements in Artificial Intelligence especially deep learning, many data-driven decision support systems have been implemented to facilitate medical doctors in delivering personalized care. We focus …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+1

Certified Policy Optimisation for Nested Causal Bandits via PAC-Bayes Risk

2026-05-28 · Tim Woydt, Paul-David Zuercher arxiv

Critical sequential decisions are rarely single-timescale: a strategic decision causally shapes the context in which every subsequent tactical choice is made; standard bandit and reinforcement-learning theory does not ca…