Learning World Graph Decompositions To Accelerate Reinforcement Learning

2019-09-25 · Wenling Shang, Alex Trott, Stephan Zheng, Caiming Xiong, Richard Socher

Efficiently learning to solve tasks in complex environments is a key challenge for reinforcement learning (RL) agents. We propose to decompose a complex environment using a task-agnostic world graphs, an abstraction that accelerates learning by enabling agents to focus exploration on a subspace of the environment.The nodes of a world graph are important waypoint states and edges represent feasible traversals between them. Our framework has two learning phases: 1) identifying world graph nodes and edges by training a binary recurrent variational auto-encoder (VAE) on trajectory data and 2) a hierarchical RL framework that leverages structural and connectivity knowledge from the learned world graph to bias exploration towards task-relevant waypoints and regions. We show that our approach significantly accelerates RL on a suite of challenging 2D grid world tasks: compared to baselines, world graph integration doubles achieved rewards on simpler tasks, e.g. MultiGoal, and manages to solve more challenging tasks, e.g. Door-Key, where baselines fail.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning Convex Decomposition via Feature Fields

2026-03-10 · Yuezhi Yang, Qixing Huang, Mikaela Angelina Uy, Nicholas Sharp arxiv

This work proposes a new formulation to the long-standing problem of convex decomposition through learning feature fields, enabling the first feed-forward model for open-world convex decomposition. Our method produces hi…

Self-Supervised Learning

Reinforcement Learning in Factored Action Spaces using Tensor Decompositions

2021-10-27 · Anuj Mahajan, Mikayel Samvelyan, Lei Mao, Viktor Makoviychuk 외

We present an extended abstract for the previously published work TESSERACT [Mahajan et al., 2021], which proposes a novel solution for Reinforcement Learning (RL) in large, factored action spaces using tensor decomposit…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Learning Sketch Decompositions in Planning via Deep Reinforcement Learning

2024-12-11 · Michael Aichmüller, Hector Geffner

In planning and reinforcement learning, the identification of common subgoal structures across problems is important when goals are to be achieved over long horizons. Recently, it has been shown that such structures can …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning

Model based Multi-agent Reinforcement Learning with Tensor Decompositions

2021-10-27 · Pascal Van Der Vaart, Anuj Mahajan, Shimon Whiteson

A challenge in multi-agent reinforcement learning is to be able to generalize over intractable state-action spaces. Inspired from Tesseract [Mahajan et al., 2021], this position paper investigates generalisation in state…

Model-based Reinforcement Learning Multi-agent Reinforcement Learning Position reinforcement-learning+2

Learning World Graphs to Accelerate Hierarchical Reinforcement Learning

2019-07-01 · Wenling Shang, Alex Trott, Stephan Zheng, Caiming Xiong 외

In many real-world scenarios, an autonomous agent often encounters various tasks within a single complex environment. We propose to build a graph abstraction over the environment structure to accelerate the learning of t…

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)