Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning

2019-05-07 · Emma Tosch, Kaleigh Clary, John Foley, David Jensen

Evaluation of deep reinforcement learning (RL) is inherently challenging. In particular, learned policies are largely opaque, and hypotheses about the behavior of deep RL agents are difficult to test in black-box environments. Considerable effort has gone into addressing opacity, but almost no effort has been devoted to producing high quality environments for experimental evaluation of agent behavior. We present TOYBOX, a new high-performance, open-source* subset of Atari environments re-designed for the experimental evaluation of deep RL. We show that TOYBOX enables a wide range of experiments and analyses that are impossible in other environments. *https://kdl-umass.github.io/Toybox/

📄 PDF Abstract BibTeX arXiv:1905.02825

Code (2)

toybox-rs/Toybox tf

toybox-rs/openai-baselines-envs tf

Tasks

Deep Reinforcement Learningreinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

ToyBox: Better Atari Environments for Testing Reinforcement Learning Agents

2018-12-06 · John Foley, Emma Tosch, Kaleigh Clary, David Jensen

It is a widely accepted principle that software without tests has bugs. Testing reinforcement learning agents is especially difficult because of the stochastic nature of both agents and environments, the complexity of st…

Atari Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

The Toybox Dataset of Egocentric Visual Object Transformations

2018-06-15 · Xiaohan Wang, Tengyu Ma, James Ainooson, Seunghwan Cha 외

In object recognition research, many commonly used datasets (e.g., ImageNet and similar) contain relatively sparse distributions of object instances and views, e.g., one might see a thousand different pictures of a thous…

Object Object Recognition Translation

RRLS : Robust Reinforcement Learning Suite

2024-06-12 · Adil Zouitine, David Bertoin, Pierre Clavier, Matthieu Geist 외

Robust reinforcement learning is the problem of learning control policies that provide optimal worst-case performance against a span of adversarial environments. It is a crucial ingredient for deploying algorithms in rea…

continuous-control Continuous Control MuJoCo reinforcement-learning+1

SocialJax: An Evaluation Suite for Multi-agent Reinforcement Learning in Sequential Social Dilemmas

2025-03-18 · Zihao Guo, Shuqing Shi, Richard Willis, Tristan Tomilin 외

Sequential social dilemmas pose a significant challenge in the field of multi-agent reinforcement learning (MARL), requiring environments that accurately reflect the tension between individual and collective interests. P…

Multi-agent Reinforcement Learning reinforcement-learning Reinforcement Learning rllib

Bench-MFG: A Benchmark Suite for Learning in Stationary Mean Field Games

2026-02-13 · Lorenzo Magnino, Jiacheng Shen, Matthieu Geist, Olivier Pietquin 외 arxiv

The intersection of Mean Field Games (MFGs) and Reinforcement Learning (RL) has fostered a growing family of algorithms designed to solve large-scale multi-agent systems. However, the field currently lacks a standardized…

Reinforcement Learning