paper-with-me

홈 › Papers

Confounding Masquerading as Improvement: A Systematic Evaluation of Offline Reinforcement Learning for Stroke Antithrombotic Treatment in a 129,000-Patient Registry

2026-08-31 · Kihun Rhee arxiv

Recent offline reinforcement learning (RL) studies report policies that outperform physician decisions on clinical outcomes. We conduct a systematic, partially crossed evaluation of five offline RL algorithm families and 14 reward designs in 44,894 post-2018 acute ischemic stroke patients from a nationwide registry (N = 129,033). Standard Fitted Q-Evaluation (FQE) yields an apparent policy-improvement estimate of +0.0069; adding an Early Neurological Deterioration penalty increases it to +0.0101. We identify reward-embedded confounding, in which a proxy terminal reward encodes baseline severity and prognosis as well as treatment efficacy. A 2 x 2 factorial analysis finds that terminal reward confounding accounts for 218.6% of the observed signal change, so its removal overshoots the null. After DML-inspired GBM reward residualization, the FQE estimate attenuates to +0.0033 (p = 0.132), and full deconfounding yields +0.0025 (p = 0.291). FQE-based diagnostics, T-learner analyses, and direct recurrence analyses converge away from a clinically meaningful aggregate improvement. A 1-year mRS factorial analysis replicates the attenuation. We provide an empirically motivated six-step evaluation checklist. NIHSS-stratified heterogeneity is hypothesis-generating for prospective trial design; hospital-level disagreement does not persist after full reward deconfounding.

📄 PDF Abstract BibTeX arXiv:2608.30442

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement LearningOffline RL

Similar Papers 제목 키워드 기반

On the Role of Similarity in Detecting Masquerading Files

2024-02-17 · Jonathan Oliver, Jue Mo, Susmit Yenkar, Raghav Batta 외

Similarity has been applied to a wide range of security applications, typically used in machine learning models. We examine the problem posed by masquerading samples; that is samples crafted by bad actors to be similar o…

Offline Policy Evaluation and Optimization under Confounding

2022-11-29 · Chinmaya Kausik, Yangyi Lu, Kevin Tan, Maggie Makar 외

Evaluating and optimizing policies in the presence of unobserved confounders is a problem of growing interest in offline reinforcement learning. Using conventional methods for offline RL in the presence of confounding ca…

Offline RLOff-policy evaluationReinforcement Learning (RL)

Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding

2023-06-01 · Alizée Pace, Hugo Yèche, Bernhard Schölkopf, Gunnar Rätsch 외

A prominent challenge of offline reinforcement learning (RL) is the issue of hidden confounding: unobserved variables may influence both the actions taken by the agent and the observed outcomes. Hidden confounding can co…

ManagementOffline RLreinforcement-learningReinforcement Learning+1

Causal Flow Q-Learning for Robust Offline Reinforcement Learning

2026-02-02 · Mingxuan Li, Junzhe Zhang, Elias Bareinboim arxiv

Expressive policies based on flow-matching have been successfully applied in reinforcement learning (RL) more recently due to their ability to model complex action distributions from offline data. These algorithms build …

Reinforcement LearningOffline RL

Session-Level Dynamic Ad Load Optimization using Offline Robust Reinforcement Learning

2025-01-09 · Tao Liu, Qi Xu, Wei Shi, Zhigang Hua 외

Session-level dynamic ad load optimization aims to personalize the density and types of delivered advertisements in real time during a user's online session by dynamically balancing user experience quality and ad monetiz…

OpenAI Gym