Many-Goals Reinforcement Learning

2018-06-22 · Vivek Veeriah, Junhyuk Oh, Satinder Singh

All-goals updating exploits the off-policy nature of Q-learning to update all possible goals an agent could have from each transition in the world, and was introduced into Reinforcement Learning (RL) by Kaelbling (1993). In prior work this was mostly explored in small-state RL problems that allowed tabular representations and where all possible goals could be explicitly enumerated and learned separately. In this paper we empirically explore 3 different extensions of the idea of updating many (instead of all) goals in the context of RL with deep neural networks (or DeepRL for short). First, in a direct adaptation of Kaelbling's approach we explore if many-goals updating can be used to achieve mastery in non-tabular visual-observation domains. Second, we explore whether many-goals updating can be used to pre-train a network to subsequently learn faster and better on a single main task of interest. Third, we explore whether many-goals updating can be used to provide auxiliary task updates in training a network to learn faster and better on a single main task of interest. We provide comparisons to baselines for each of the 3 extensions.

📄 PDF Abstract BibTeX arXiv:1806.09605

Code (0)

등록된 구현이 없습니다.

Tasks

AllQ-Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Methods 이 논문이 사용한 방법론

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

The Max-Min Formulation of Multi-Objective Reinforcement Learning: From Theory to a Model-Free Algorithm

2024-06-12 · Giseung Park, Woohyeon Byeon, Seongmin Kim, Elad Havakuk 외

In this paper, we consider multi-objective reinforcement learning, which arises in many real-world problems with multiple optimization goals. We approach the problem with a max-min framework focusing on fairness among th…

Fairness Multi-Objective Reinforcement Learning reinforcement-learning Reinforcement Learning

Vision-Language Models as a Source of Rewards

2023-12-14 · Kate Baumli, Satinder Baveja, Feryal Behbahani, Harris Chan 외

Building generalist agents that can accomplish many goals in rich open-ended environments is one of the research frontiers for reinforcement learning. A key limiting factor for building generalist agents with RL has been…

reinforcement-learning Reinforcement Learning

Explainable Reinforcement Learning Through Goal-Based Explanations

2021-01-01 · Gregory Bonaert, Youri Coppens, Denis Steckelmacher, Ann Nowe

Many algorithms in Reinforcement Learning rely on neural networks to achieve state-of-the-art performance, but this has the cost of making the agents black-boxes, hard to interpret and understand, making their use diffic…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learning

2019-01-24 · Sanjeevan Ahilan, Peter Dayan

We investigate how reinforcement learning agents can learn to cooperate. Drawing inspiration from human societies, in which successful coordination of many individuals is often facilitated by hierarchical organisation, w…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Clustering via Dirichlet Process Mixture Models for Portable Skill Discovery

2011-12-01 · NeurIPS 2011 12 · Scott Niekum, Andrew G. Barto

Skill discovery algorithms in reinforcement learning typically identify single states or regions in state space that correspond to task-specific subgoals. However, such methods do not directly address the question of ho…

Clustering Reinforcement Learning Reinforcement Learning (RL)