Dynamic Weights in Multi-Objective Deep Reinforcement Learning

2018-09-20 · Axel Abels, Diederik M. Roijers, Tom Lenaerts, Ann Nowé, Denis Steckelmacher

Many real-world decision problems are characterized by multiple conflicting objectives which must be balanced based on their relative importance. In the dynamic weights setting the relative importance changes over time and specialized algorithms that deal with such change, such as a tabular Reinforcement Learning (RL) algorithm by Natarajan and Tadepalli (2005), are required. However, this earlier work is not feasible for RL settings that necessitate the use of function approximators. We generalize across weight changes and high-dimensional inputs by proposing a multi-objective Q-network whose outputs are conditioned on the relative importance of objectives and we introduce Diverse Experience Replay (DER) to counter the inherent non-stationarity of the Dynamic Weights setting. We perform an extensive experimental evaluation and compare our methods to adapted algorithms from Deep Multi-Task/Multi-Objective Reinforcement Learning and show that our proposed network in combination with DER dominates these adapted algorithms across weight change scenarios and problem domains.

📄 PDF Abstract BibTeX arXiv:1809.07803

Code (3)

axelabels/DynMORL 공식 구현

axelabels/Minecart

lucasalegre/mo-gym

Tasks

Deep Reinforcement LearningMulti-Objective Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

Learning to Optimize Multi-Objective Alignment Through Dynamic Reward Weighting

2025-09-14 · Yining Lu, Zilong Wang, Shiyang Li, Xin Liu 외 arxiv

Prior work in multi-objective reinforcement learning typically uses linear reward scalarization with fixed weights, which provably fails to capture non-convex Pareto fronts and thus yields suboptimal results. This limita…

Reinforcement Learning

Flexible Multiple-Objective Reinforcement Learning for Chip Placement

2022-04-13 · Fu-Chieh Chang, Yu-Wei Tseng, Ya-Wen Yu, Ssu-Rui Lee 외

Recently, successful applications of reinforcement learning to chip placement have emerged. Pretrained models are necessary to improve efficiency and effectiveness. Currently, the weights of objective metrics (e.g., wire…

Diversity reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Multi-timestep models for Model-based Reinforcement Learning

2023-10-09 · Abdelhakim Benechehab, Giuseppe Paolo, Albert Thomas, Maurizio Filippone 외

In model-based reinforcement learning (MBRL), most algorithms rely on simulating trajectories from one-step dynamics models learned on data. A critical challenge of this approach is the compounding of one-step prediction…

model Model-based Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Multiobjective Reinforcement Learning for Reconfigurable Adaptive Optimal Control of Manufacturing Processes

2018-09-18 · Johannes Dornheim, Norbert Link

In industrial applications of adaptive optimal control often multiple contrary objectives have to be considered. The weights (relative importance) of the objectives are often not known during the design of the control an…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

PSMGD: Periodic Stochastic Multi-Gradient Descent for Fast Multi-Objective Optimization

2024-12-14 · Mingjing Xu, Peizhong Ju, Jia Liu, Haibo Yang

Multi-objective optimization (MOO) lies at the core of many machine learning (ML) applications that involve multiple, potentially conflicting objectives (e.g., multi-task learning, multi-objective reinforcement learning,…

Multi-Objective Reinforcement Learning Multi-Task Learning