Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination

2024-05-28 · Zhiyao Luo, Yangchen Pan, Peter Watkinson, Tingting Zhu

In the rapidly changing healthcare landscape, the implementation of offline reinforcement learning (RL) in dynamic treatment regimes (DTRs) presents a mix of unprecedented opportunities and challenges. This position paper offers a critical examination of the current status of offline RL in the context of DTRs. We argue for a reassessment of applying RL in DTRs, citing concerns such as inconsistent and potentially inconclusive evaluation metrics, the absence of naive and supervised learning baselines, and the diverse choice of RL formulation in existing research. Through a case study with more than 17,000 evaluation experiments using a publicly available Sepsis dataset, we demonstrate that the performance of RL algorithms can significantly vary with changes in evaluation metrics and Markov Decision Process (MDP) formulations. Surprisingly, it is observed that in some instances, RL algorithms can be surpassed by random baselines subjected to policy evaluation methods and reward design. This calls for more careful policy evaluation and algorithm development in future DTR works. Additionally, we discussed potential enhancements toward more reliable development of RL-based dynamic treatment regimes and invited further discussion within the community. Code is available at https://github.com/GilesLuo/ReassessDTR.

📄 PDF Abstract BibTeX arXiv:2405.18556

Code (1)

gilesluo/reassessdtr 공식 구현 pytorch

Tasks

Offline RL reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Deep Reinforcement Learning for Dynamic Treatment Regimes on Medical Registry Data

2018-01-28 · Ning Liu, Ying Liu, Brent Logan, Zhiyuan Xu 외

This paper presents the first deep reinforcement learning (DRL) framework to estimate the optimal Dynamic Treatment Regimes from observational medical data. This framework is more flexible and adaptive for high dimension…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimes

2024-06-29 · Sophia Yazzourh, Nicolas Savy, Philippe Saint-Pierre, Michael R. Kosorok

The goal of precision medicine is to provide individualized treatment at each stage of chronic diseases, a concept formalized by Dynamic Treatment Regimes (DTR). These regimes adapt treatment strategies based on decision…

reinforcement-learning Reinforcement Learning (RL)

Designing Optimal Dynamic Treatment Regimes: A Causal Reinforcement Learning Approach

2020-01-01 · ICML 2020 1 · Junzhe Zhang

A dynamic treatment regime (DTR) consists of a sequence of decision rules, one per stage of intervention, that dictates how to determine the treatment assignment to patients based on evolving treatments and covariates' h…

Decision Making reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approach

2021-12-08 · Soroush Saghafian

A main research goal in various studies is to use an observational data set and provide a new set of counterfactual guidelines that can yield causal improvements. Dynamic Treatment Regimes (DTRs) are widely studied to fo…

counterfactual Decision Making reinforcement-learning Reinforcement Learning+1

Near-Optimal Reinforcement Learning in Dynamic Treatment Regimes

2019-12-01 · NeurIPS 2019 12 · Junzhe Zhang, Elias Bareinboim