Diversity Induced Environment Design via Self-Play

2023-02-04 · Dexun Li, Wenjun Li, Pradeep Varakantham

Recent work on designing an appropriate distribution of environments has shown promise for training effective generally capable agents. Its success is partly because of a form of adaptive curriculum learning that generates environment instances (or levels) at the frontier of the agent's capabilities. However, such an environment design framework often struggles to find effective levels in challenging design spaces and requires costly interactions with the environment. In this paper, we aim to introduce diversity in the Unsupervised Environment Design (UED) framework. Specifically, we propose a task-agnostic method to identify observed/hidden states that are representative of a given level. The outcome of this method is then utilized to characterize the diversity between two levels, which as we show can be crucial to effective performance. In addition, to improve sampling efficiency, we incorporate the self-play technique that allows the environment generator to automatically generate environments that are of great benefit to the training agent. Quantitatively, our approach, Diversity-induced Environment Design via Self-Play (DivSP), shows compelling performance over existing methods.

📄 PDF Abstract BibTeX arXiv:2302.02119

Code (0)

등록된 구현이 없습니다.

Tasks

Diversity

The Essence of Balance for Self-Improving Agents in Vision-and-Language Navigation

2026-04-21 · Zhen Liu, Yuhan Liu, Jinjun Wang, Jianyi Liu 외 arxiv

In vision-and-language navigation (VLN), self-improvement from policy-induced experience, using only standard VLN action supervision, critically depends on balancing behavioral diversity and learning stability, which gov…

Enhanced Generalization through Prioritization and Diversity in Self-Imitation Reinforcement Learning over Procedural Environments with Sparse Rewards

2023-11-01 · Alain Andres, Daochen Zha, Javier Del Ser

Exploration poses a fundamental challenge in Reinforcement Learning (RL) with sparse rewards, limiting an agent's ability to learn optimal decision-making due to a lack of informative feedback signals. Self-Imitation Lea…

Decision Making Diversity Imitation Learning Reinforcement Learning (RL)

The built environment and induced transport CO2 emissions: A double machine learning approach to account for residential self-selection

2023-12-07 · Florian Nachtigall, Felix Wagner, Peter Berrill, Felix Creutzig

Understanding why travel behavior differs between residents of urban centers and suburbs is key to sustainable urban planning. Especially in light of rapid urban growth, identifying housing locations that minimize travel…

Hidden incentives for self-induced distributional shift

2019-09-25 · David Scott Krueger, Tegan Maharaj, Shane Legg, Jan Leike

Decisions made by machine learning systems have increasing influence on the world. Yet it is common for machine learning algorithms to assume that no such influence exists. An example is the use of the i.i.d. assumption …

BIG-bench Machine Learning Meta-Learning

Learning Diverse Risk Preferences in Population-based Self-play

2023-05-19 · Yuhua Jiang, Qihan Liu, Xiaoteng Ma, Chenghao Li 외

Among the great successes of Reinforcement Learning (RL), self-play algorithms play an essential role in solving competitive games. Current self-play algorithms optimize the agent to maximize expected win-rates against i…

Diversity reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)