홈 › Papers

The Role of Pretrained Representations for the OOD Generalization of Reinforcement Learning Agents

2021-07-12 · ICLR 2022 4 · Andrea Dittadi, Frederik Träuble, Manuel Wüthrich, Felix Widmaier, Peter Gehler, Ole Winther, Francesco Locatello, Olivier Bachem, Bernhard Schölkopf, Stefan Bauer

Building sample-efficient agents that generalize out-of-distribution (OOD) in real-world settings remains a fundamental unsolved problem on the path towards achieving higher-level cognition. One particularly promising approach is to begin with low-dimensional, pretrained representations of our world, which should facilitate efficient downstream learning and generalization. By training 240 representations and over 10,000 reinforcement learning (RL) policies on a simulated robotic setup, we evaluate to what extent different properties of pretrained VAE-based representations affect the OOD generalization of downstream agents. We observe that many agents are surprisingly robust to realistic distribution shifts, including the challenging sim-to-real case. In addition, we find that the generalization performance of a simple downstream proxy task reliably predicts the generalization performance of our RL agents under a wide range of OOD settings. Such proxy tasks can thus be used to select pretrained representations that will lead to agents that generalize.

📄 PDF Abstract BibTeX arXiv:2107.05686

Code (0)

등록된 구현이 없습니다.

Tasks

Reinforcement Learning (RL)Representation Learning

Pretrained Encoders are All You Need

2021-06-09 · ICML Workshop URL 2021 7 · Mina Khan, P Srivatsa, Advait Rane, Shriram Chenniappa 외

Data-efficiency and generalization are key challenges in deep learning and deep reinforcement learning as many models are trained on large-scale, domain-specific, and expensive-to-label datasets. Self-supervised models t…

All Contrastive Learning Deep Reinforcement Learning reinforcement-learning+2

MORAL: A Multimodal Reinforcement Learning Framework for Decision Making in Autonomous Laboratories

2025-04-04 · Natalie Tirabassi, Sathish A. P. Kumar, Sumit Jha, Arvind Ramanathan

We propose MORAL (a multimodal reinforcement learning framework for decision making in autonomous laboratories) that enhances sequential decision-making in autonomous robotic laboratories through the integration of visua…

Decision Making Image Captioning Language Modeling Language Modelling+3

CORD: Generalizable Cooperation via Role Diversity

2025-01-04 · Kanefumi Matsuyama, Kefan Su, Jiangxing Wang, Deheng Ye 외

Cooperative multi-agent reinforcement learning (MARL) aims to develop agents that can collaborate effectively. However, most cooperative MARL methods overfit training agents, making learned policies not generalize well t…

Diversity Multi-agent Reinforcement Learning

DMC-VB: A Benchmark for Representation Learning for Control with Visual Distractors

2024-09-26 · Joseph Ortiz, Antoine Dedieu, Wolfgang Lehrach, Swaroop Guntupalli 외

Learning from previously collected data via behavioral cloning or offline reinforcement learning (RL) is a powerful recipe for scaling generalist agents by avoiding the need for expensive online learning. Despite strong …

continuous-control Continuous Control Offline RL Reinforcement Learning (RL)+1

Eye of the Beholder: Improved Relation Generalization for Text-based Reinforcement Learning Agents

2021-06-09 · Keerthiram Murugesan, Subhajit Chaudhury, Kartik Talamadupula

Text-based games (TBGs) have become a popular proving ground for the demonstration of learning-based agents that make decisions in quasi real-world settings. The crux of the problem for a reinforcement learning agent in …

reinforcement-learning Reinforcement Learning (RL)Relation text-based games