Neural Population Learning beyond Symmetric Zero-sum Games
We study computationally efficient methods for finding equilibria in n-player general-sum games, specifically ones that afford complex visuomotor skills. We show how existing methods would struggle in this setting, either computationally or in theory. We then introduce NeuPL-JPSRO, a neural population learning algorithm that benefits from transfer learning of skills and converges to a Coarse Correlated Equilibrium (CCE) of the game. We show empirical convergence in a suite of OpenSpiel games, validated rigorously by exact game solvers. We then deploy NeuPL-JPSRO to complex domains, where our approach enables adaptive coordination in a MuJoCo control domain and skill transfer in capture-the-flag. Our work shows that equilibrium convergent population learning can be implemented at scale and in generality, paving the way towards solving real-world games between heterogeneous players with mixed motives.
Code (0)
등록된 구현이 없습니다.
Tasks
MuJoCoTransfer LearningSimilar Papers 제목 키워드 기반
MF-OML: Online Mean-Field Reinforcement Learning with Occupation Measures for Large Population Games
Reinforcement learning for multi-agent games has attracted lots of attention recently. However, given the challenge of solving Nash equilibria for large population games, existing works with guaranteed polynomial complex…
Multi-agent Reinforcement Learningreinforcement-learningReinforcement LearningOpen-ended Learning in Symmetric Zero-sum Games
Zero-sum games such as chess and poker are, abstractly, functions that evaluate pairs of agents, for example labeling them `winner' and `loser'. If the game is approximately transitive, then self-play generates sequences…
Structural symmetry in evolutionary games
In evolutionary game theory, an important measure of a mutant trait (strategy) is its ability to invade and take over an otherwise-monomorphic population. Typically, one quantifies the success of a mutant strategy via th…
Real World Games Look Like Spinning Tops
This paper investigates the geometrical properties of real world games (e.g. Tic-Tac-Toe, Go, StarCraft II). We hypothesise that their geometrical structure resemble a spinning top, with the upright axis representing tra…
ClusteringStarcraftStarcraft IIDynamical stability of evolutionarily stable strategy in asymmetric games
Evolutionarily stable strategy (ESS) is the defining concept of evolutionary game theory. It has a fairly unanimously accepted definition for the case of symmetric games which are played in a homogeneous population where…