Demonstration Guided Multi-Objective Reinforcement Learning
Multi-objective reinforcement learning (MORL) is increasingly relevant due to its resemblance to real-world scenarios requiring trade-offs between multiple objectives. Catering to diverse user preferences, traditional reinforcement learning faces amplified challenges in MORL. To address the difficulty of training policies from scratch in MORL, we introduce demonstration-guided multi-objective reinforcement learning (DG-MORL). This novel approach utilizes prior demonstrations, aligns them with user preferences via corner weight support, and incorporates a self-evolving mechanism to refine suboptimal demonstrations. Our empirical studies demonstrate DG-MORL's superiority over existing MORL algorithms, establishing its robustness and efficacy, particularly under challenging conditions. We also provide an upper bound of the algorithm's sample complexity.
Code (0)
등록된 구현이 없습니다.
Tasks
Multi-Objective Reinforcement Learningreinforcement-learningReinforcement LearningSimilar Papers 제목 키워드 기반
Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learning
While Reinforcement Learning (RL) has been proven essential for tuning large language models (LLMs), it can lead to reward over-optimization (ROO). Existing approaches address ROO by adding KL regularization, requiring c…
Reinforcement Learning (RL)Text GenerationReinforcement Learning via Implicit Imitation Guidance
We study the problem of sample efficient reinforcement learning, where prior data such as demonstrations are provided for initialization in lieu of a dense reward signal. A natural approach is to incorporate an imitation…
continuous-controlContinuous ControlImitation Learningreinforcement-learning+1Demonstration-Guided Reinforcement Learning with Learned Skills
Demonstration-guided reinforcement learning (RL) is a promising approach for learning complex behaviors by leveraging both reward feedback and a set of target task demonstrations. Prior approaches for demonstration-guide…
reinforcement-learningReinforcement LearningReinforcement Learning (RL)Robot ManipulationGPU-Parallel Multi-Task Reinforcement Learning with Demonstration Guided Policy Optimization
Large scale GPU-parallel reinforcement learning has changed what can be trained in robot simulation, yet most systems still optimize one specialist policy per task. We propose a construction methodology for turning struc…
Reinforcement LearningAn Offline Adaptation Framework for Constrained Multi-Objective Reinforcement Learning
In recent years, significant progress has been made in multi-objective reinforcement learning (RL) research, which aims to balance multiple objectives by incorporating preferences for each objective. In most existing stu…
Multi-Objective Reinforcement LearningReinforcement Learning (RL)