Modelling the Dynamics of Multiagent Q-Learning in Repeated Symmetric Games: a Mean Field Theoretic Approach
Modelling the dynamics of multi-agent learning has long been an important research topic, but all of the previous works focus on 2-agent settings and mostly use evolutionary game theoretic approaches. In this paper, we study an n-agent setting with n tends to infinity, such that agents learn their policies concurrently over repeated symmetric bimatrix games with some other agents. Using mean field theory, we approximate the effects of other agents on a single agent by an averaged effect. A Fokker-Planck equation that describes the evolution of the probability distribution of Q-values in the agent population is derived. To the best of our knowledge, this is the first time to show the Q-learning dynamics under an n-agent setting can be described by a system of only three equations. We validate our model through comparisons with agent-based simulations on typical symmetric bimatrix games and different initial settings of Q-values.
Code (0)
등록된 구현이 없습니다.
Tasks
Q-LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Evolutionary Game-Theoretical Analysis for General Multiplayer Asymmetric Games
Evolutionary game theory has been a successful tool to combine classical game theory with learning-dynamical descriptions in multiagent systems. Provided some symmetric structures of interacting players, many studies hav…
StarcraftStarcraft IITwo-Player Incomplete Games of Resilient Multiagent Systems
Evolution of agents' dynamics of multiagent systems under consensus protocol in the face of jamming attacks is discussed, where centralized parties are able to influence the control signals of the agents. In this paper w…
Vocal Bursts Valence PredictionSA-IGA: A Multiagent Reinforcement Learning Method Towards Socially Optimal Outcomes
In multiagent environments, the capability of learning is important for an agent to behave appropriately in face of unknown opponents and dynamic environment. From the system designer's perspective, it is desirable if th…
Q-Learningreinforcement-learningReinforcement LearningReinforcement Learning (RL)On the Convergence of No-Regret Learning Dynamics in Time-Varying Games
Most of the literature on learning in games has focused on the restrictive setting where the underlying repeated game does not change over time. Much less is known about the convergence of no-regret learning algorithms i…
Meta-LearningAsymmetric Equilibria in Symmetric Multiplayer Prisoners Dilemma Supergames
We propose a finite automaton-style solution concept for supergames. In our model, we define an equilibrium to be a cycle of state switches and a supergame to be an infinite walk on states of a finite stage game. We show…