Prediction-Guided Multi-Objective Reinforcement Learning for Continuous Robot Control
Many real-world control problems involve conflicting objectives where we desire a dense and high-quality set of control policies that are optimal for different objective preferences (called Pareto-optimal). While extensive research in multi-objective reinforcement learning (MORL) has been conducted to tackle such problems, multi-objective optimization for complex continuous robot control is still under-explored. In this work, we propose an efficient evolutionary learning algorithm to find the Pareto set approximation for continuous robot control problems, by extending a state-of-the-art RL algorithm and presenting a novel prediction model to guide the learning process. In addition to efficiently discovering the individual policies on the Pareto front, we construct a continuous set of Pareto-optimal solutions by Pareto analysis and interpolation. Furthermore, we design six multi-objective RL environments with continuous action space, which is the first benchmark platform to evaluate MORL algorithms on various robot control problems. We test the previous methods on the proposed benchmark problems, and the experiments show that our approach is able to find a much denser and higher-quality set of Pareto policies than the existing algorithms.
Code (2)
Tasks
Multi-Objective Reinforcement Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting
Prior work in multi-objective reinforcement learning typically uses linear reward scalarization with fixed weights, which provably fails to capture non-convex Pareto fronts and thus yields suboptimal results. This limita…
Reinforcement LearningDemonstration Guided Multi-Objective Reinforcement Learning
Multi-objective reinforcement learning (MORL) is increasingly relevant due to its resemblance to real-world scenarios requiring trade-offs between multiple objectives. Catering to diverse user preferences, traditional re…
Multi-Objective Reinforcement Learningreinforcement-learningReinforcement LearningA proof of convergence of inverse reinforcement learning for multi-objective optimization
We show the convergence of Wasserstein inverse reinforcement learning for multi-objective optimizations with the projective subgradient method by formulating an inverse problem of the multi-objective optimization problem…
reinforcement-learningReinforcement LearningRTDK-BO: High Dimensional Bayesian Optimization with Reinforced Transformer Deep kernels
Bayesian Optimization (BO), guided by Gaussian process (GP) surrogates, has proven to be an invaluable technique for efficient, high-dimensional, black-box optimization, a critical problem inherent to many applications s…
Bayesian OptimizationMeta-Learningreinforcement-learningReinforcement Learning+1MOMA-AC: A preference-driven actor-critic framework for continuous multi-objective multi-agent reinforcement learning
This paper addresses a critical gap in Multi-Objective Multi-Agent Reinforcement Learning (MOMARL) by introducing the first dedicated inner-loop actor-critic framework for continuous state and action spaces: Multi-Object…
Multi-agent Reinforcement Learning