Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation

2018-06-28 · Niels Justesen, Ruben Rodriguez Torrado, Philip Bontrager, Ahmed Khalifa, Julian Togelius, Sebastian Risi

Deep reinforcement learning (RL) has shown impressive results in a variety of domains, learning directly from high-dimensional sensory streams. However, when neural networks are trained in a fixed environment, such as a single level in a video game, they will usually overfit and fail to generalize to new levels. When RL models overfit, even slight modifications to the environment can result in poor agent performance. This paper explores how procedurally generated levels during training can increase generality. We show that for some games procedural level generation enables generalization to new levels within the same distribution. Additionally, it is possible to achieve better performance with less data by manipulating the difficulty of the levels in response to the performance of the agent. The generality of the learned behaviors is also evaluated on a set of human-designed levels. The results suggest that the ability to generalize to human-designed levels highly depends on the design of the level generators. We apply dimensionality reduction and clustering techniques to visualize the generators' distributions of levels and analyze to what degree they can produce levels similar to those designed by a human.

📄 PDF Abstract BibTeX arXiv:1806.10729

Code (1)

njustesen/a2c_gvgai 공식 구현 tf

Tasks

ClusteringDeep Reinforcement LearningDimensionality Reduction reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Illuminating the Space of Dungeon Maps, Locked-door Missions and Enemy Placement Through MAP-Elites

2022-02-18 · Breno M. F. Viana, Leonardo T. Pereira, Claudio F. M. Toledo

Procedural Content Generation (PCG) methods are valuable tools to speed up the game development process. Moreover, PCG may also present in games as features, such as the procedural dungeon generation (PDG) in Moonlighter…

Demonstration-efficient Inverse Reinforcement Learning in Procedurally Generated Environments

2020-12-04 · Alessandro Sestini, Alexander Kuhnle, Andrew D. Bagdanov

Deep Reinforcement Learning achieves very good results in domains where reward functions can be manually engineered. At the same time, there is growing interest within the community in using games based on Procedurally C…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Procedural Game Level Design with Deep Reinforcement Learning

2025-10-16 · Miraç Buğra Özkan arxiv

Procedural content generation (PCG) has become an increasingly popular technique in game development, allowing developers to generate dynamic, replayable, and scalable environments with reduced manual effort. In this stu…

Reinforcement Learning

Leveraging Procedural Generation to Benchmark Reinforcement Learning

2019-12-03 · ICML 2020 1 · Karl Cobbe, Christopher Hesse, Jacob Hilton, John Schulman

We introduce Procgen Benchmark, a suite of 16 procedurally generated game-like environments designed to benchmark both sample efficiency and generalization in reinforcement learning. We believe that the community will be…

Procgen Hard (100M)reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Adversarial Reinforcement Learning for Procedural Content Generation

2021-03-08 · Linus Gisslén, Andy Eakins, Camilo Gordillo, Joakim Bergdahl 외

We present a new approach ARLPCG: Adversarial Reinforcement Learning for Procedural Content Generation, which procedurally generates and tests previously unseen environments with an auxiliary input as a control variable.…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)