Tit for Tattling: Cooperation, communication, and how each could stabilize the other
Indirect reciprocity is a mechanism by which individuals cooperate with those who have cooperated with others. This creates a regime in which repeated interactions are not necessary to incent cooperation (as would be required for direct reciprocity). However, indirect reciprocity creates a new problem: how do agents know who has cooperated with others? To know this, agents would need to access some form of reputation information. Perhaps there is a communication system to disseminate reputation information, but how does it remain truthful and informative? Most papers assume the existence of a truthful, forthcoming, and informative communication system; in this paper, we seek to explain how such a communication system could remain evolutionarily stable in the absence of exogenous pressures. Specifically, we present three conditions that together maintain both the truthfulness of the communication system and the prevalence of cooperation: individuals (1) use a norm that rewards the behaviors that it prescribes (an aligned norm), (2) can signal not only about the actions of other agents, but also about their truthfulness (by acting as third party observers to an interaction), and (3) make occasional mistakes, demonstrating how error can create stability by introducing diversity.
Code (0)
등록된 구현이 없습니다.
Tasks
DiversitySimilar Papers 제목 키워드 기반
Learning Attentional Communication for Multi-Agent Cooperation
Communication could potentially be an effective way for multi-agent cooperation. However, information sharing among all agents or in predefined communication architectures that existing methods adopt can be problematic. …
Decision MakingCan maker-taker fees prevent algorithmic cooperation in market making?
In a semi-realistic market simulator, independent reinforcement learning algorithms may facilitate market makers to maintain wide spreads even without communication. This unexpected outcome challenges the current antitru…
reinforcement-learningReinforcement Learning (RL)Behavioral Mistakes Support Cooperation in an N-Person Repeated Public Goods Game
This study investigates the effect of behavioral mistakes on the evolutionary stability of the cooperative equilibrium in a repeated public goods game. Many studies show that behavioral mistakes have detrimental effects …
Learning Individually Inferred Communication for Multi-Agent Cooperation
Communication lays the foundation for human cooperation. It is also crucial for multi-agent cooperation. However, existing work focuses on broadcast communication, which is not only impractical but also leads to informat…
Causal InferenceMulti-agent Reinforcement LearningWho's Driving? Game Theoretic Path Risk of AGI Development
Who controls the development of Artificial General Intelligence (AGI) might matter less than how we handle the fight for control itself. We formalize this "steering wheel problem" as humanity's greatest near-term existen…