OraPO: Oracle-educated Reinforcement Learning for Data-efficient and Factual Radiology Report Generation
Radiology report generation (RRG) aims to automatically produce clinically faithful reports from chest X-ray images. Prevailing work typically follows a scale-driven paradigm, by multi-stage training over large paired corpora and oversized backbones, making pipelines highly data- and compute-intensive. In this paper, we propose Oracle-educated GRPO (OraPO) with a FactScore-based reward (FactS) to tackle the RRG task under constrained budgets. OraPO enables single-stage, RL-only training by converting failed GRPO explorations on rare or difficult studies into direct preference supervision via a lightweight oracle step. FactS grounds learning in diagnostic evidence by extracting atomic clinical facts and checking entailment against ground-truth labels, yielding dense, interpretable sentence-level rewards. Together, OraPO and FactS create a compact and powerful framework that significantly improves learning efficiency on clinically challenging cases, setting the new SOTA performance on the CheXpert Plus dataset (0.341 in F1) with 2--3 orders of magnitude less training data using a small base VLM on modest hardware.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningSimilar Papers 제목 키워드 기반
Opinion-Guided Reinforcement Learning
Human guidance is often desired in reinforcement learning to improve the performance of the learning agent. However, human insights are often mere opinions and educated guesses rather than well-formulated arguments. Whil…
Efficient Explorationreinforcement-learningReinforcement LearningQuantum oracles give an advantage for identifying classical counterfactuals
We show that quantum oracles provide an advantage over classical oracles for answering classical counterfactual questions in causal models, or equivalently, for identifying unknown causal parameters such as distributions…
Education Policy and Intergenerational Educational Persistence: Evidence from rural Benin
This paper employs a nonlinear difference-in-differences approach to empirically examine the Maximally Maintained Inequality (MMI) hypothesis in rural Benin. The findings of this study confirm the MMI hypothesis. In part…
Counterfactual Copula and Its Application to the Effects of College Education on Intergenerational Mobility
This paper proposes a nonparametric estimator of the counterfactual copula of two outcome variables that would be affected by a policy intervention. The proposed estimator allows policymakers to conduct ex-ante evaluatio…
counterfactualAgentic Search for Counterfactual Recourse under Fixed LLM Budgets
Counterfactual recourse aims to provide actionable feature changes that would alter an unfavorable decision made by a predictive model. In practice, affected individuals often benefit from multiple feasible alternatives …