Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agents
There has been rapidly growing interest in meta-learning as a method for increasing the flexibility and sample efficiency of reinforcement learning. One problem in this area of research, however, has been a scarcity of adequate benchmark tasks. In general, the structure underlying past benchmarks has either been too simple to be inherently interesting, or too ill-defined to support principled analysis. In the present work, we introduce a new benchmark for meta-RL research, emphasizing transparency and potential for in-depth analysis as well as structural richness. Alchemy is a 3D video game, implemented in Unity, which involves a latent causal structure that is resampled procedurally from episode to episode, affording structure learning, online inference, hypothesis testing and action sequencing based on abstract domain knowledge. We evaluate a pair of powerful RL agents on Alchemy and present an in-depth analysis of one of these agents. Results clearly indicate a frank and specific failure of meta-learning, providing validation for Alchemy as a challenging benchmark for meta-RL. Concurrent with this report, we are releasing Alchemy as public resource, together with a suite of analysis tools and sample agent trajectories.
Code (1)
Tasks
Meta-LearningMeta Reinforcement Learningreinforcement-learningReinforcement Learning (RL)UnitySimilar Papers 제목 키워드 기반
A model-based approach to meta-Reinforcement Learning: Transformers and tree search
Meta-learning is a line of research that develops the ability to leverage past experiences to efficiently solve new learning problems. Meta-Reinforcement Learning (meta-RL) methods demonstrate a capability to learn behav…
Meta-LearningMeta Reinforcement Learningreinforcement-learningReinforcement Learning (RL)How to Learn and Represent Abstractions: An Investigation using Symbolic Alchemy
Alchemy is a new meta-learning environment rich enough to contain interesting abstractions, yet simple enough to make fine-grained analysis tractable. Further, Alchemy provides an optional symbolic interface that enables…
Meta-LearningBioAlchemy: Distilling Biological Literature into Reasoning-Ready Reinforcement Learning Training Data
Despite the large corpus of biology training text, the impact of reasoning models on biological research generally lags behind math and coding. In this work, we show that biology questions from current large-scale reason…
Reinforcement LearningHypothesis Network Planned Exploration for Rapid Meta-Reinforcement Learning Adaptation
Meta Reinforcement Learning (Meta RL) trains agents that adapt to fast-changing environments and tasks. Current strategies often lose adaption efficiency due to the passive nature of model exploration, causing delayed un…
Meta Reinforcement Learningreinforcement-learningReinforcement LearningAlchemy: A Quantum Chemistry Dataset for Benchmarking AI Models
We introduce a new molecular dataset, named Alchemy, for developing machine learning models useful in chemistry and material science. As of June 20th 2019, the dataset comprises of 12 quantum mechanical properties of 119…
BenchmarkingBIG-bench Machine LearningDiversityGraph Neural Network