Learning Pareto Set for Multi-Objective Continuous Robot Control

2024-06-27 · Tianye Shu, Ke Shang, Cheng Gong, Yang Nan, Hisao Ishibuchi

For a control problem with multiple conflicting objectives, there exists a set of Pareto-optimal policies called the Pareto set instead of a single optimal policy. When a multi-objective control problem is continuous and complex, traditional multi-objective reinforcement learning (MORL) algorithms search for many Pareto-optimal deep policies to approximate the Pareto set, which is quite resource-consuming. In this paper, we propose a simple and resource-efficient MORL algorithm that learns a continuous representation of the Pareto set in a high-dimensional policy parameter space using a single hypernet. The learned hypernet can directly generate various well-trained policy networks for different user preferences. We compare our method with two state-of-the-art MORL algorithms on seven multi-objective continuous robot control problems. Experimental results show that our method achieves the best overall performance with the least training parameters. An interesting observation is that the Pareto set is well approximated by a curved line or surface in a high-dimensional parameter space. This observation will provide insight for researchers to design new MORL algorithms.

📄 PDF Abstract BibTeX arXiv:2406.18924

Code (1)

hisaolabsustc/hyper-morl 공식 구현 tf

Tasks

Multi-Objective Reinforcement Learning

Methods 이 논문이 사용한 방법론

SET Dynamic Sparse Training method where weight mask is updated randomly periodically

Prediction-Guided Multi-Objective Reinforcement Learning for Continuous Robot Control

2020-01-01 · ICML 2020 1 · Jie Xu, Yunsheng Tian, Pingchuan Ma, Daniela Rus 외

Many real-world control problems involve conflicting objectives where we desire a dense and high-quality set of control policies that are optimal for different objective preferences (called Pareto-optimal). While extensi…

Multi-Objective Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

PA2D-MORL: Pareto Ascent Directional Decomposition based Multi-Objective Reinforcement Learning

2026-03-20 · Tianmeng Hu, Biao Luo arxiv

Multi-objective reinforcement learning (MORL) provides an effective solution for decision-making problems involving conflicting objectives. However, achieving high-quality approximations to the Pareto policy set remains …

Reinforcement Learning

Post Hoc Extraction of Pareto Fronts for Continuous Control

2026-03-03 · Raghav Thakar, Gaurav Dixit, Kagan Tumer arxiv

Agents in the real world must often balance multiple objectives, such as speed, stability, and energy efficiency in continuous control. To account for changing conditions and preferences, an agent must ideally learn a Pa…

Reinforcement Learning Continuous Control

Pareto Frontier Approximation Network (PA-Net) Applied to Multi-objective TSP

2021-09-29 · Ishaan Mehta, Sajad Saeedi

Multi-objective optimization is used in various areas of robotics like control, planning etc. Their solutions are dependent on multiple objective functions, which can be conflicting in nature. In such cases, the optimali…

A Safe Reinforcement Learning driven Weights-varying Model Predictive Control for Autonomous Vehicle Motion Control

2024-02-04 · Baha Zarrouki, Marios Spanakakis, Johannes Betz

Determining the optimal cost function parameters of Model Predictive Control (MPC) to optimize multiple control objectives is a challenging and time-consuming task. Multiobjective Bayesian Optimization (BO) techniques so…

Bayesian Optimization Deep Reinforcement Learning Model Predictive Control Reinforcement Learning (RL)+1