Sampled Policy Gradient for Learning to Play the Game Agar.io

2018-09-15 · Anton Orell Wiehe, Nil Stolt Ansó, Madalina M. Drugan, Marco A. Wiering

In this paper, a new offline actor-critic learning algorithm is introduced: Sampled Policy Gradient (SPG). SPG samples in the action space to calculate an approximated policy gradient by using the critic to evaluate the samples. This sampling allows SPG to search the action-Q-value space more globally than deterministic policy gradient (DPG), enabling it to theoretically avoid more local optima. SPG is compared to Q-learning and the actor-critic algorithms CACLA and DPG in a pellet collection task and a self play environment in the game Agar.io. The online game Agar.io has become massively popular on the internet due to intuitive game design and the ability to instantly compete against players around the world. From the point of view of artificial intelligence this game is also very intriguing: The game has a continuous input and action space and allows to have diverse agents with complex strategies compete against each other. The experimental results show that Q-Learning and CACLA outperform a pre-programmed greedy bot in the pellet collection task, but all algorithms fail to outperform this bot in a fighting scenario. The SPG algorithm is analyzed to have great extendability through offline exploration and it matches DPG in performance even in its basic form without extensive sampling.

📄 PDF Abstract BibTeX arXiv:1809.05763

Code (2)

RUKip/MachineLearningProject

mrcabo/MachineLearningProject

Tasks

Game DesignQ-Learning

Methods 이 논문이 사용한 방법론

DPG Deterministic Policy Gradient, or DPG, is a policy gradient method for reinforcement learning. Instead of the policy function $\pi\left(.\mid{s}\right)$ being modeled as a…

Q-Learning Q-Learning is an off-policy temporal difference control algorithm: $$Q\left(S\_{t}, A\_{t}\right) \leftarrow Q\left(S\_{t}, A\_{t}\right) + \alpha\left[R_{t+1} +…

Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomization

2021-03-08 · ICLR 2021 1 · Zhenggang Tang, Chao Yu, Boyuan Chen, Huazhe Xu 외

We propose a simple, general and effective technique, Reward Randomization for discovering diverse strategic policies in complex multi-agent games. Combining reward randomization and policy gradient, we derive a new algo…

The Cell Must Go On: Agar.io for Continual Reinforcement Learning

2025-05-23 · Mohamed A. Mohamed, Kateryna Nekhomiazh, Vedant Vyas, Marcos M. Jose 외

Continual reinforcement learning (RL) concerns agents that are expected to learn continually, rather than converge to a policy that is then fixed for evaluation. Such an approach is well suited to environments the agent …

Continual Learning Deep Reinforcement Learning Reinforcement Learning (RL)

Data-Augmented Game Starts for Accelerating Self-Play Exploration in Imperfect Information Games

2026-05-14 · JB Lanier, Nathan Monette, Pierre Baldi, Roy Fox arxiv

Finding approximate equilibria for large-scale imperfect-information competitive games such as StarCraft, Dota, and CounterStrike remains computationally infeasible due to sparse rewards and challenging exploration over …

Reinforcement Learning

Adversarial Policy Gradient for Alternating Markov Games

2018-01-01 · ICLR 2018 1 · Chao Gao, Martin Mueller, Ryan Hayward

Policy gradient reinforcement learning has been applied to two-player alternate-turn zero-sum games, e.g., in AlphaGo, self-play REINFORCE was used to improve the neural net model after supervised learning. In this paper…

Policy Gradient Methods Reinforcement Learning

Policy-Gradient Algorithms Have No Guarantees of Convergence in Linear Quadratic Games

2019-07-08 · Eric Mazumdar, Lillian J. Ratliff, Michael. I. Jordan, S. Shankar Sastry

We show by counterexample that policy-gradient algorithms have no guarantees of even local convergence to Nash equilibria in continuous action and state space multi-agent settings. To do so, we analyze gradient-play in N…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)