Quantifying Generalization in Reinforcement Learning
In this paper, we investigate the problem of overfitting in deep reinforcement learning. Among the most common benchmarks in RL, it is customary to use the same environments for both training and testing. This practice offers relatively little insight into an agent's ability to generalize. We address this issue by using procedurally generated environments to construct distinct training and test sets. Most notably, we introduce a new environment called CoinRun, designed as a benchmark for generalization in RL. Using CoinRun, we find that agents overfit to surprisingly large training sets. We then show that deeper convolutional architectures improve generalization, as do methods traditionally found in supervised learning, including L2 regularization, dropout, data augmentation and batch normalization.
Code (1)
Tasks
Data AugmentationDeep Reinforcement LearningL2 Regularizationreinforcement-learningReinforcement LearningReinforcement Learning (RL)Similar Papers 제목 키워드 기반
Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalization
Generalization poses a significant challenge in Multi-agent Reinforcement Learning (MARL). The extent to which an agent is influenced by unseen co-players depends on the agent's policy and the specific scenario. A quanti…
Multi-agent Reinforcement LearningQuantifying the effects of environment and population diversity in multi-agent reinforcement learning
Generalization is a major challenge for multi-agent reinforcement learning. How well does an agent perform when placed in novel environments and in interactions with new co-players? In this paper, we investigate and quan…
DiversityMulti-agent Reinforcement Learningreinforcement-learningReinforcement Learning (RL)Quantifying and Optimizing Simplicity via Polynomial Representations
Deep networks often exhibit a preference for "simple" solutions, and such a simplicity bias is widely believed to play a key role in generalization. Yet a broadly applicable, quantitative measure of simplicity remains el…
Reinforcement LearningText ClassificationReinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting
Reinforcement learning (RL) is a promising approach for aligning large language models (LLMs) knowledge with sequential decision-making tasks. However, few studies have thoroughly investigated the impact on LLM agents ca…
Decision MakingReinforcement Learning (RL)SensitivitySequential Decision MakingCoBA-RL: Capability-Oriented Budget Allocation for Reinforcement Learning in LLMs
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a key approach for enhancing LLM reasoning. However, standard frameworks like Group Relative Policy Optimization (GRPO) typically employ a uniform roll…
Reinforcement Learning