The Cell Must Go On: Agar.io for Continual Reinforcement Learning
Continual reinforcement learning (RL) concerns agents that are expected to learn continually, rather than converge to a policy that is then fixed for evaluation. Such an approach is well suited to environments the agent perceives as changing, which renders any static policy ineffective over time. The few simulators explicitly designed for empirical research in continual RL are often limited in scope or complexity, and it is now common for researchers to modify episodic RL environments by artificially incorporating abrupt task changes during interaction. In this paper, we introduce AgarCL, a research platform for continual RL that allows for a progression of increasingly sophisticated behaviour. AgarCL is based on the game Agar.io, a non-episodic, high-dimensional problem featuring stochastic, ever-evolving dynamics, continuous actions, and partial observability. Additionally, we provide benchmark results reporting the performance of DQN, PPO, and SAC in both the primary, challenging continual RL problem, and across a suite of smaller tasks within AgarCL, each of which isolates aspects of the full environment and allow us to characterize the challenges posed by different aspects of the game.
Code (1)
Tasks
Continual LearningDeep Reinforcement LearningReinforcement Learning (RL)Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Biophysical Model for Signal-Embedded Droplet Soaking into 2D Cell Culture
Using agar plates hosting a 2D cell population stimulated with signaling molecules is crucial for experiments such as gene regulation and drug discovery in a wide range of biological studies. In this paper, a biophysical…
Cultural Vocal Bursts Intensity PredictionDrug DiscoveryPAGAR: Taming Reward Misalignment in Inverse Reinforcement Learning-Based Imitation Learning with Protagonist Antagonist Guided Adversarial Reward
Many imitation learning (IL) algorithms employ inverse reinforcement learning (IRL) to infer the intrinsic reward function that an expert is implicitly optimizing for based on their demonstrated behaviors. However, in pr…
Imitation LearningZero-Shot LearningContinual Model-based Reinforcement Learning for Data Efficient Wireless Network Optimisation
We present a method that addresses the pain point of long lead-time required to deploy cell-level parameter optimisation policies to new wireless network sites. Given a sequence of action spaces represented by overlappin…
Model-based Reinforcement LearningA Short Note on Soft-max and Policy Gradients in Bandits Problems
This is a short communication on a Lyapunov function argument for softmax in bandit problems. There are a number of excellent papers coming out using differential equations for policy gradient algorithms in reinforcement…
reinforcement-learningReinforcement Learning (RL)Task diversity produces systematic transfer but inhibits continual reinforcement learning
Continual reinforcement learning aims to produce agents that learn not only to improve at their current tasks but also to adapt as task distributions change. Training an agent on many diverse tasks can induce zero-shot g…
Zero-shot GeneralizationReinforcement LearningContinual Learning