Learning Strategic Value and Cooperation in Multi-Player Stochastic Games through Side Payments
For general-sum, n-player, strategic games with transferable utility, the Harsanyi-Shapley value provides a computable method to both 1) quantify the strategic value of a player; and 2) make cooperation rational through side payments. We give a simple formula to compute the HS value in normal-form games. Next, we provide two methods to generalize the HS values to stochastic (or Markov) games, and show that one of them may be computed using generalized Q-learning algorithms. Finally, an empirical validation is performed on stochastic grid-games with three or more players. Source code is provided to compute HS values for both the normal-form and stochastic game setting.
Code (0)
등록된 구현이 없습니다.
Tasks
FormQ-LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Evolution of Cooperation in Public Goods Games with Stochastic Opting-Out
This paper investigates the evolution of strategic play where players drawn from a finite well-mixed population are offered the opportunity to play in a public goods game. All players accept the offer. However, due to th…
Multiplayer Bandit Learning, from Competition to Cooperation
The stochastic multi-armed bandit model captures the tradeoff between exploration and exploitation. We study the effects of competition and cooperation on this tradeoff. Suppose there are $k$ arms and two players, Alice …
Comparing reactive and memory-one strategies of direct reciprocity
Direct reciprocity is a mechanism for the evolution of cooperation based on repeated interactions. When individuals meet repeatedly, they can use conditional strategies to enforce cooperative outcomes that would not be f…
Noisy information channel mediated prevention of the tragedy of the commons
Synergy between evolutionary dynamics of cooperation and fluctuating state of shared resource being consumed by the cooperators is essential for averting the tragedy of the commons. Not only in humans, but also in the co…
Network topology and movement cost, not updating mechanism, determine the evolution of cooperation in mobile structured populations
Evolutionary models are used to study the self-organisation of collective action, often incorporating population structure due to its ubiquitous presence and long-known impact on emerging phenomena. We investigate the ev…