Papers Offline RL
“Offline RL” 태그가 달린 논문 996편 · 필터 해제
Towards Optimal Policy Improvement
Practical Reinforcement Learning (RL) algorithms learn to solve Markov Decision Processes (MDPs) through iterative policy improvement in the presence of approximate evaluation. We study policy improvement from first prin…
Reinforcement LearningOffline RLDecision Titan: Test-Time Training for Long-Term Memory in Offline Reinforcement Learning
Long-term dependencies remain a major challenge for sequential decision-making in the field of AI: RNNs suffer from vanishing gradients and the limited expressivity of vector-based hidden states, whilst Transformer-based…
Reinforcement LearningOffline RLX-Tree: Tokenizing Reusable Experience for Efficient Agent Generalization
Multi-step agents are trained on flat action streams: SFT and RLVR weight every token uniformly and ignore the sub-procedures that recur across tasks, the hierarchy that lets humans plan top-down from reusable routines. …
Offline RLConfounding Masquerading as Improvement: A Systematic Evaluation of Offline Reinforcement Learning for Stroke Antithrombotic Treatment in a 129,000-Patient Registry
Recent offline reinforcement learning (RL) studies report policies that outperform physician decisions on clinical outcomes. We conduct a systematic, partially crossed evaluation of five offline RL algorithm families and…
Reinforcement LearningOffline RLLearning to Allocate Incentives for Incentivized Advertising via Offline Model-Based Reinforcement Learning
Complete your ad view and grab a 5-cent bonus! In incentivized advertising, a platform promises users a bonus before observing downstream ad revenue, encouraging them to click and complete ads. It must balance the incent…
Reinforcement LearningCausal InferenceOffline RLSimple Actors and Deep Critics for Scalable Reinforcement Learning
Recent progress in offline reinforcement learning (RL) has been driven by expressive generative actors such as diffusion and flow-matching policies, which capture multimodal behavior in offline datasets. However, these a…
Reinforcement LearningOffline RLDecoupling Policy Extraction for Offline Reinforcement Learning
Offline RL methods commonly jointly train the actor and critic, where the critic is used to guide the actor toward higher-value actions. This coupled learning process is well motivated in online RL, where an improved act…
Reinforcement LearningOffline RLRoMAN-Flow: Taming Autoregressive Normalizing Flows for Offline Reinforcement Learning in Robotic Manipulation
Offline reinforcement learning improves robotic policies using previously collected data without further environment interaction. Yet prevalent diffusion- and flow-matching robot policies lack tractable likelihoods, limi…
Reinforcement LearningOffline RLOffline Deep Q* Estimation with Diffusion Models
In offline RL, estimating the optimal action-value function $Q^*$ can be formulated as solving the optimal Bellman equation based solely on offline observations. A fundamental challenge is that the reward function and tr…
Offline RLTraining a Conditioned Video Game Agent on a VLM Annotated Dataset
Reinforcement Learning (RL) is a powerful but far from easy-to-use technique for policy learning. In the specific case of video games, access to the game engine is required to get rewards for training (e.g. to collect re…
Reinforcement LearningOffline RLConvex-Hull-Neighborhood Smooth Dual Generalization: Controlling Local Correction Propagation in Offline RL
Offline reinforcement learning (offline RL) can benefit from nearby out-of-distribution (OOD) actions, but estimation errors at these actions may be amplified by bootstrapping. Existing regularization and local-generaliz…
Reinforcement LearningOffline RLRMSWeb: Reflection, Failure-Mode Mining, and Salvage-DS for Web Agent Reinforcement Learning
Compact web agents can reduce deployment cost, but training them poses challenges in both data collection and post-SFT reinforcement learning (RL). Successful trajectories are expensive to collect and often contain ineff…
Reinforcement LearningOffline RLRedFlow: Redirect Failure into Action-Level Corrections for Flow-matching VLA Policy
Flow-matching Vision-Language-Action (VLA) policies have shown strong potential for robotic manipulation but often suffer from compounding errors caused by distribution shifts during deployment. While offline reinforceme…
Reinforcement LearningOffline RLRL^2-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models
Despite the impressive visuomotor capabilities enabled by Vision-Language-Action (VLA) models, their performance often degrades on challenging and out-of-domain tasks. Recent test-time steering and scaling methods improv…
Reinforcement LearningOffline RLLearning from the Unseen: Offline Reinforcement Learning with Hidden Actions
Standard offline reinforcement learning (RL) algorithms typically assume that the actions in the dataset are observed without error. However, in many real-world applications, the true actions are unobserved and only nois…
Reinforcement LearningOffline RLTOUR: A Trajectory-Level Unlearning Benchmark for Offline Reinforcement Learning
Offline Reinforcement Learning (RL) agents are trained on fixed behavioral trajectories, which makes trajectory-level deletion important when selected data must be removed after training. Evaluating such deletion is diff…
Reinforcement LearningOffline RLOffline RL with Hierarchical Action Chunking
Offline goal-conditioned reinforcement learning (RL) holds the promise of learning general-purpose policies from static datasets. However, scaling these methods to long-horizon tasks remains a challenge due to the curse …
Reinforcement LearningOffline RLThe Mechanism Matters: When Knowledge Graphs Help Reinforcement Learning
Knowledge graphs (KGs) are widely used to inject prior knowledge into reinforcement learning (RL), yet the literature is dominated by single-domain, positive-result method papers, so we lack a systematic account of when …
Reinforcement LearningKnowledge GraphsOffline RLEvaluating covariate balance for long time horizon Markov decision processes
This article explores the application of covariate balance diagnostics for detecting the presence of hidden confounding/model miss-specification in studies applying offline reinforcement learning (RL) to deriving optimal…
Reinforcement LearningOffline RLVINE: Taming Generative Control Policies for Reinforcement Learning
Flow-matching policies have emerged as an effective policy parameterization for robot learning. They iteratively generate actions from noise, enabling highly expressive modeling of complex and multimodal action distribut…
Reinforcement LearningOffline RL