Solving single-objective tasks by preference multi-objective reinforcement learning

2019-09-25 · Jinsheng Ren, Shangqi Guo, Feng Chen

There ubiquitously exist many single-objective tasks in the real world that are inevitably related to some other objectives and influenced by them. We call such task as the objective-constrained task, which is inherently a multi-objective problem. Due to the conflict among different objectives, a trade-off is needed. A common compromise is to design a scalar reward function through clarifying the relationship among these objectives using the prior knowledge of experts. However, reward engineering is extremely cumbersome. This will result in behaviors that optimize our reward function without actually satisfying our preferences. In this paper, we explicitly cast the objective-constrained task as preference multi-objective reinforcement learning, with the overall goal of finding a Pareto optimal policy. Combined with Trajectory Preference Domination we propose, a weight vector that reflects the agent's preference for each objective can be learned. We analyzed the feasibility of our algorithm in theory, and further proved in experiments its better performance compared to those that design the reward function by experts.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-Objective Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Controllable Pareto Multi-Task Learning

2020-10-13 · Xi Lin, Zhiyuan Yang, Qingfu Zhang, Sam Kwong

A multi-task learning (MTL) system aims at solving multiple related tasks at the same time. With a fixed model capacity, the tasks would be conflicted with each other, and the system usually has to make a trade-off among…

Multiobjective Optimization Multi-Task Learning

A Single Deep Preference-Conditioned Policy for Learning Pareto Coverage Sets

2026-05-09 · Akihiro Kubo, Kosuke Nakanishi, Shin Ishii arxiv

Preference-conditioned multi-objective reinforcement learning aims to learn a single policy that captures trade-offs across preferences, but under nonlinear scalarization the uniqueness and continuity of the preference-t…

Reinforcement Learning

Policy-regularized Offline Multi-objective Reinforcement Learning

2024-01-04 · Qian Lin, Chao Yu, Zongkai Liu, Zifan Wu

In this paper, we aim to utilize only offline trajectory data to train a policy for multi-objective RL. We extend the offline policy-regularized method, a widely-adopted approach for single-objective offline RL problems,…

Multi-Objective Reinforcement Learning Offline RL reinforcement-learning Reinforcement Learning

Pareto Multi-Task Learning

2019-12-30 · NeurIPS 2019 12 · Xi Lin, Hui-Ling Zhen, Zhenhua Li, Qingfu Zhang 외

Multi-task learning is a powerful method for solving multiple correlated tasks simultaneously. However, it is often impossible to find one single solution to optimize all the tasks, since different tasks might conflict w…

Multiobjective Optimization Multi-Task Learning

Pareto Set Learning for Neural Multi-objective Combinatorial Optimization

2022-03-29 · Xi Lin, Zhiyuan Yang, Qingfu Zhang

Multiobjective combinatorial optimization (MOCO) problems can be found in many real-world applications. However, exactly solving these problems would be very challenging, particularly when they are NP-hard. Many handcraf…

Combinatorial Optimization Traveling Salesman Problem