Defining Admissible Rewards for High Confidence Policy Evaluation
A key impediment to reinforcement learning (RL) in real applications with limited, batch data is defining a reward function that reflects what we implicitly know about reasonable behaviour for a task and allows for robust off-policy evaluation. In this work, we develop a method to identify an admissible set of reward functions for policies that (a) do not diverge too far from past behaviour, and (b) can be evaluated with high confidence, given only a collection of past trajectories. Together, these ensure that we propose policies that we trust to be implemented in high-risk settings. We demonstrate our approach to reward design on synthetic domains as well as in a critical care context, for a reward that consolidates clinical objectives to learn a policy for weaning patients from mechanical ventilation.
Code (0)
등록된 구현이 없습니다.
Tasks
Off-policy evaluationReinforcement LearningReinforcement Learning (RL)Vocal Bursts Intensity PredictionSimilar Papers 제목 키워드 기반
Distributionally Robust Markov Decision Processes
We consider Markov decision processes where the values of the parameters are uncertain. This uncertainty is described by a sequence of nested sets (that is, each set contains the previous one), each of which corresponds …
Anchored Policy Optimization: Mitigating Exploration Collapse Via Support-Constrained Rectification
Reinforcement Learning with Verifiable Rewards (RLVR) is increasingly viewed as a tree pruning mechanism. However, we identify a systemic pathology termed Recursive Space Contraction (RSC), an irreversible collapse drive…
Reinforcement LearningTriviality Corrected Endogenous Reward
Reinforcement learning for open-ended text generation is constrained by the lack of verifiable rewards, necessitating reliance on judge models that require either annotated data or powerful closed-source models. Inspired…
Reinforcement LearningMathematical ReasoningText GenerationRobust Shielding for Safe Reinforcement Learning
Shielding is an effective approach to formally guarantee the safety of reinforcement learning agents in Markov decision processes (MDPs). However, existing shielding techniques typically assume knowledge of the safety-re…
Reinforcement LearningConfidence-Controlled Exploration: Efficient Sparse-Reward Policy Learning for Robot Navigation
Reinforcement learning (RL) is a promising approach for robotic navigation, allowing robots to learn through trial and error. However, real-world robotic tasks often suffer from sparse rewards, leading to inefficient exp…
Policy Gradient Methodsreinforcement-learningReinforcement Learning (RL)Robot Navigation