Generalizing Across Multi-Objective Reward Functions in Deep Reinforcement Learning

2018-09-17 · Eli Friedman, Fred Fontaine

Many reinforcement-learning researchers treat the reward function as a part of the environment, meaning that the agent can only know the reward of a state if it encounters that state in a trial run. However, we argue that this is an unnecessary limitation and instead, the reward function should be provided to the learning algorithm. The advantage is that the algorithm can then use the reward function to check the reward for states that the agent hasn't even encountered yet. In addition, the algorithm can simultaneously learn policies for multiple reward functions. For each state, the algorithm would calculate the reward using each of the reward functions and add the rewards to its experience replay dataset. The Hindsight Experience Replay algorithm developed by Andrychowicz et al. (2017) does just this, and learns to generalize across a distribution of sparse, goal-based rewards. We extend this algorithm to linearly-weighted, multi-objective rewards and learn a single policy that can generalize across all linear combinations of the multi-objective reward. Whereas other multi-objective algorithms teach the Q-function to generalize across the reward weights, our algorithm enables the policy to generalize, and can thus be used with continuous actions.

📄 PDF Abstract BibTeX arXiv:1809.06364

Code (1)

trality/fire

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Experience Replay Experience Replay is a replay memory technique used in reinforcement learning where we store the agent’s experiences at each time-step, $e\_{t} = \left(s\_{t}, a\_{t}, r\_{t},…

On Generalization Across Environments In Multi-Objective Reinforcement Learning

2025-03-02 · Jayden Teoh, Pradeep Varakantham, Peter Vamplew

Real-world sequential decision-making tasks often require balancing trade-offs between multiple conflicting objectives, making Multi-Objective Reinforcement Learning (MORL) an increasingly prominent field of research. De…

Decision Making Multi-Objective Reinforcement Learning reinforcement-learning Reinforcement Learning+1

MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimization

2024-02-18 · Yasaman Jafari, Dheeraj Mekala, Rose Yu, Taylor Berg-Kirkpatrick

RL-based techniques can be employed to search for prompts that, when fed into a target language model, maximize a set of user-specified reward functions. However, in many target applications, the natural reward functions…

Language Modelling Machine Translation Multi-Objective Reinforcement Learning Style Transfer

Automatic Reward Shaping from Multi-Objective Human Heuristics

2025-12-17 · Yuqing Xie, Jiayu Chen, Wenhao Tang, Ya Zhang 외 arxiv

Designing effective reward functions remains a central challenge in reinforcement learning, especially in multi-objective environments. In this work, we propose Multi-Objective Reward Shaping with Exploration (MORSE), a …

Reinforcement Learning

GIFT: Generalizing Intent for Flexible Test-Time Rewards

2026-03-23 · Fin Amin, Nathaniel Dennler, Andreea Bobu arxiv

Robots learn reward functions from user demonstrations, but these rewards often fail to generalize to new environments. This failure occurs because learned rewards latch onto spurious correlations in training data rather…

Semantic Similarity

Generalizing distribution of partial rewards for multi-armed bandits with temporally-partitioned rewards

2022-11-13 · Ronald C. van den Broek, Rik Litjens, Tobias Sagis, Luc Siecker 외

We investigate the Multi-Armed Bandit problem with Temporally-Partitioned Rewards (TP-MAB) setting in this paper. In the TP-MAB setting, an agent will receive subsets of the reward over multiple rounds rather than the en…

Multi-Armed Bandits