Reward-Conditioned Reinforcement Learning
Single-task RL agents are typically trained under a fixed reward function, which limits their robustness to reward misspecification and their ability to adapt to changing preferences. We introduce Reward-Conditioned Reinforcement Learning (RCRL), an off-policy method that conditions agents on reward parameterizations while collecting experience under a single nominal objective. By recomputing counterfactual rewards from shared replay data, RCRL exposes the agent to multiple reward objectives without additional environment interaction, connecting single-task RL with ideas from multi-objective and multi-task learning. Across single-task, multi-task, and vision-based benchmarks, RCRL improves sample efficiency under the nominal reward parameterization, enables efficient adaptation to new parameterizations, and supports zero-shot behavioral adjustment at deployment. Our results show that RCRL provides a scalable mechanism for learning robust, steerable policies without sacrificing the simplicity of single-task training.
Code (0)
등록된 구현이 없습니다.
Tasks
Reinforcement LearningMulti-Task LearningSimilar Papers 제목 키워드 기반
Why Goal-Conditioned Reinforcement Learning Works: Relation to Dual Control
Goal-conditioned reinforcement learning (RL) concerns the problem of training an agent to maximize the probability of reaching target goal states. This paper presents an analysis of the goal-conditioned setting based on …
Reinforcement LearningFrom Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following
Reinforcement learning is a promising framework for solving control problems, but its use in practical situations is hampered by the fact that reward functions are often difficult to engineer. Specifying goals and tasks …
Instruction Followingreinforcement-learningReinforcement LearningReinforcement Learning (RL)Phasic Self-Imitative Reduction for Sparse-Reward Goal-Conditioned Reinforcement Learning
It has been a recent trend to leverage the power of supervised learning (SL) towards more effective reinforcement learning (RL) methods. We propose a novel phasic approach by alternating online RL and offline SL for tack…
reinforcement-learningReinforcement Learning (RL)Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularization
Reinforcement learning (RL) methods with a high replay ratio (RR) and regularization have gained interest due to their superior sample efficiency. However, these methods have mainly been developed for dense-reward tasks.…
Q-LearningReinforcement Learning (RL)Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learning
It is of significance for an agent to learn a widely applicable and general-purpose policy that can achieve diverse goals including images and text descriptions. Considering such perceptually-specific goals, the frontier…
Deep Reinforcement Learningreinforcement-learningReinforcement Learning (RL)