On Reinforcement Learning for Full-length Game of StarCraft

2018-09-23 · Zhen-Jia Pang, Ruo-Ze Liu, Zhou-Yu Meng, Yi Zhang, Yang Yu, Tong Lu

StarCraft II poses a grand challenge for reinforcement learning. The main difficulties of it include huge state and action space and a long-time horizon. In this paper, we investigate a hierarchical reinforcement learning approach for StarCraft II. The hierarchy involves two levels of abstraction. One is the macro-action automatically extracted from expert's trajectories, which reduces the action space in an order of magnitude yet remains effective. The other is a two-layer hierarchical architecture which is modular and easy to scale, enabling a curriculum transferring from simpler tasks to more complex tasks. The reinforcement training algorithm for this architecture is also investigated. On a 64x64 map and using restrictive units, we achieve a winning rate of more than 99\% against the difficulty level-1 built-in AI. Through the curriculum transfer learning algorithm and a mixture of combat model, we can achieve over 93\% winning rate of Protoss against the most difficult non-cheating built-in AI (level-7) of Terran, training within two days using a single machine with only 48 CPU cores and 8 K40 GPUs. It also shows strong generalization performance, when tested against never seen opponents including cheating levels built-in AI and all levels of Zerg and Protoss built-in AI. We hope this study could shed some light on the future research of large-scale reinforcement learning.

📄 PDF Abstract BibTeX arXiv:1809.09095

Code (0)

등록된 구현이 없습니다.

Tasks

CPUHierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)StarcraftStarcraft II Transfer Learning

Reinforcement Learning for the Beginning of Starcraft II Game

2020-12-14 · CUHK Course IERG5350 2020 12 · Yukang Chen, Ruihang Chu

Starcraft II is a popular real-time strategy game that is welcomed by many young people. This game is really complicated to play due to the long game timeline, various society/buildings/units, a large number of actions o…

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)Starcraft+1

SCC: an efficient deep reinforcement learning agent mastering the game of StarCraft II

2020-12-24 · Xiangjun Wang, Junxiao Song, Penghui Qi, Peng Peng 외

AlphaStar, the AI that reaches GrandMaster level in StarCraft II, is a remarkable milestone demonstrating what deep reinforcement learning can achieve in complex Real-Time Strategy (RTS) games. However, the complexities …

Deep Reinforcement Learning Imitation Learning reinforcement-learning Reinforcement Learning+3

On Efficient Reinforcement Learning for Full-length Game of StarCraft II

2022-09-23 · Ruo-Ze Liu, Zhen-Jia Pang, Zhou-Yu Meng, Wenhai Wang 외

StarCraft II (SC2) poses a grand challenge for reinforcement learning (RL), of which the main difficulties include huge state space, varying action space, and a long time horizon. In this work, we investigate a set of RL…

CPU reinforcement-learning Reinforcement Learning (RL)Starcraft+2

TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game

2018-09-19 · Peng Sun, Xinghai Sun, Lei Han, Jiechao Xiong 외

Starcraft II (SC2) is widely considered as the most challenging Real Time Strategy (RTS) game. The underlying challenges include a large observation space, a huge (continuous and infinite) action space, partial observati…

AI Agent Decision Making Deep Reinforcement Learning Real-Time Strategy Games+3

StarCraft II: A New Challenge for Reinforcement Learning

2017-08-16 · Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev 외

This paper introduces SC2LE (StarCraft II Learning Environment), a reinforcement learning environment based on the StarCraft II game. This domain poses a new grand challenge for reinforcement learning, representing a mor…

Deep Reinforcement Learning Real-Time Strategy Games reinforcement-learning Reinforcement Learning+3