Solving Sokoban using Hierarchical Reinforcement Learning with Landmarks

2025-04-06 · Sergey Pastukhov

We introduce a novel hierarchical reinforcement learning (HRL) framework that performs top-down recursive planning via learned subgoals, successfully applied to the complex combinatorial puzzle game Sokoban. Our approach constructs a six-level policy hierarchy, where each higher-level policy generates subgoals for the level below. All subgoals and policies are learned end-to-end from scratch, without any domain knowledge. Our results show that the agent can generate long action sequences from a single high-level call. While prior work has explored 2-3 level hierarchies and subgoal-based planning heuristics, we demonstrate that deep recursive goal decomposition can emerge purely from learning, and that such hierarchies can scale effectively to hard puzzle domains.

📄 PDF Abstract BibTeX arXiv:2504.04366

Code (1)

omikad/halfweg 공식 구현 pytorch

Tasks

Hierarchical Reinforcement Learning reinforcement-learning Reinforcement LearningSokoban

Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learning

2021-10-26 · NeurIPS 2021 12 · Junsu Kim, Younggyo Seo, Jinwoo Shin

Goal-conditioned hierarchical reinforcement learning (HRL) has shown promising results for solving complex and long-horizon RL tasks. However, the action space of high-level policy in the goal-conditioned HRL is often la…

Efficient Exploration Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+1

Potential-based Reward Shaping in Sokoban

2021-09-10 · Zhao Yang, Mike Preuss, Aske Plaat

Learning to solve sparse-reward reinforcement learning problems is difficult, due to the lack of guidance towards the goal. But in some problems, prior knowledge can be used to augment the learning process. Reward shapin…

Sokoban

Solving Sokoban with forward-backward reinforcement learning

2021-05-05 · Yaron Shoham, Gal Elidan

Despite seminal advances in reinforcement learning in recent years, many domains where the rewards are sparse, e.g. given only at task completion, remain quite challenging. In such cases, it can be beneficial to tackle t…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Sokoban

Beyond Tabula-Rasa: a Modular Reinforcement Learning Approach for Physically Embedded 3D Sokoban

2020-10-03 · Peter Karkus, Mehdi Mirza, Arthur Guez, Andrew Jaegle 외

Intelligent robots need to achieve abstract objectives using concrete, spatiotemporally complex sensory information and motor control. Tabula rasa deep reinforcement learning (RL) has tackled demanding tasks in terms of …

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning (RL)Sokoban

Transfer Learning and Curriculum Learning in Sokoban

2021-05-25 · Zhao Yang, Mike Preuss, Aske Plaat

Transfer learning can speed up training in machine learning and is regularly used in classification tasks. It reuses prior knowledge from other tasks to pre-train networks for new tasks. In reinforcement learning, learni…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Sokoban+1