AlphaStar Unplugged: Large-Scale Offline Reinforcement Learning
StarCraft II is one of the most challenging simulated reinforcement learning environments; it is partially observable, stochastic, multi-agent, and mastering StarCraft II requires strategic planning over long time horizons with real-time low-level execution. It also has an active professional competitive scene. StarCraft II is uniquely suited for advancing offline RL algorithms, both because of its challenging nature and because Blizzard has released a massive dataset of millions of StarCraft II games played by human players. This paper leverages that and establishes a benchmark, called AlphaStar Unplugged, introducing unprecedented challenges for offline reinforcement learning. We define a dataset (a subset of Blizzard's release), tools standardizing an API for machine learning methods, and an evaluation protocol. We also present baseline agents, including behavior cloning, offline variants of actor-critic and MuZero. We improve the state of the art of agents using only offline data, and we achieve 90% win rate against previously published AlphaStar behavior cloning agent.
Code (1)
Tasks
Offline RLreinforcement-learningReinforcement LearningStarcraftStarcraft IIMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
RL Unplugged: A Collection of Benchmarks for Offline Reinforcement Learning
Offline methods for reinforcement learning have a potential to help bridge the gap between reinforcement learning research and real-world applications. They make it possible to learn policies from offline datasets, thus …
Offline RLreinforcement-learningReinforcement LearningReinforcement Learning (RL)RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning
Offline methods for reinforcement learning have a potential to help bridge the gap between reinforcement learning research and real-world applications. They make it possible to learn policies from offline datasets, thus …
Atari GamesDQN Replay DatasetMuJoCo GamesOffline RL+3Online and Offline Reinforcement Learning by Planning with a Learned Model
Learning efficiently from small amounts of data has long been the focus of model-based reinforcement learning, both for the online case when interacting with the environment and the offline case when learning from a fixe…
Atari GamesContinuous ControlModel-based Reinforcement LearningOffline RL+3Efficient Offline Policy Optimization with a Learned Model
MuZero Unplugged presents a promising approach for offline policy learning from logged data. It conducts Monte-Carlo Tree Search (MCTS) with a learned model and leverages Reanalyze algorithm to learn purely from offline …
Offline RLAn Introduction of mini-AlphaStar
StarCraft II (SC2) is a real-time strategy game in which players produce and control multiple units to fight against opponent's units. Due to its difficulties, such as huge state space, various action space, a long time …
StarcraftStarcraft II