Efficient Reinforcement Learning for StarCraft by Abstract Forward Models and Transfer Learning

2019-03-02 · Ruo-Ze Liu, Haifeng Guo, Xiaozhong Ji, Yang Yu, Zhen-Jia Pang, Zitai Xiao, Yuzhou Wu, Tong Lu

Injecting human knowledge is an effective way to accelerate reinforcement learning (RL). However, these methods are underexplored. This paper presents our discovery that an abstract forward model (thought-game (TG)) combined with transfer learning (TL) is an effective way. We take StarCraft II as our study environment. With the help of a designed TG, the agent can learn a 99% win-rate on a 64x64 map against the Level-7 built-in AI, using only 1.08 hours in a single commercial machine. We also show that the TG method is not as restrictive as it was thought to be. It can work with roughly designed TGs, and can also be useful when the environment changes. Comparing with previous model-based RL, we show TG is more effective. We also present a TG hypothesis that gives the influence of different fidelity levels of TG. For real games that have unequal state and action spaces, we proposed a novel XfrNet of which usefulness is validated while achieving a 90% win-rate against the cheating Level-10 AI. We argue that the TG method might shed light on further studies of efficient RL with human knowledge.

📄 PDF Abstract BibTeX arXiv:1903.00715

Code (1)

liuruoze/thought-sc2-model-experiment 공식 구현 tf

Tasks

reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)StarcraftStarcraft II Transfer Learning

On Reinforcement Learning for Full-length Game of StarCraft

2018-09-23 · Zhen-Jia Pang, Ruo-Ze Liu, Zhou-Yu Meng, Yi Zhang 외

StarCraft II poses a grand challenge for reinforcement learning. The main difficulties of it include huge state and action space and a long-time horizon. In this paper, we investigate a hierarchical reinforcement learnin…

CPU Hierarchical Reinforcement Learning reinforcement-learning Reinforcement Learning+4

StarCraft Micromanagement with Reinforcement Learning and Curriculum Transfer Learning

2018-04-03 · Kun Shao, Yuanheng Zhu, Dongbin Zhao

Real-time strategy games have been an important field of game artificial intelligence in recent years. This paper presents a reinforcement learning and curriculum transfer learning method to control multiple units in Sta…

Real-Time Strategy Games reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+2

Asynchronous Advantage Actor-Critic Agent for Starcraft II

2018-07-22 · Basel Alghanem, Keerthana P G

Deep reinforcement learning, and especially the Asynchronous Advantage Actor-Critic algorithm, has been successfully used to achieve super-human performance in a variety of video games. Starcraft II is a new challenge fo…

Deep Reinforcement Learning reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)+3

Variational Offline Multi-agent Skill Discovery

2024-05-26 · Jiayu Chen, Bhargav Ganguly, Tian Lan, Vaneet Aggarwal

Skills are effective temporal abstractions established for sequential decision making, which enable efficient hierarchical learning for long-horizon tasks and facilitate multi-task learning through their transferability.…

Decision Making Multi-agent Reinforcement Learning Multi-Task Learning Sequential Decision Making+1

Predictive Representations for Skill Transfer in Reinforcement Learning

2026-04-08 · Ruben Vereecken, Luke Dickens, Alessandra Russo arxiv

A key challenge in scaling up Reinforcement Learning is generalizing learned behaviour. Without the ability to carry forward acquired knowledge an agent is doomed to learn each task from scratch. In this paper we develop…

Reinforcement Learning