Werewolf Arena: A Case Study in LLM Evaluation via Social Deduction
This paper introduces Werewolf Arena, a novel framework for evaluating large language models (LLMs) through the lens of the classic social deduction game, Werewolf. In Werewolf Arena, LLMs compete against each other, navigating the game's complex dynamics of deception, deduction, and persuasion. The framework introduces a dynamic turn-taking system based on bidding, mirroring real-world discussions where individuals strategically choose when to speak. We demonstrate the framework's utility through an arena-style tournament featuring Gemini and GPT models. Our results reveal distinct strengths and weaknesses in the models' strategic reasoning and communication. These findings highlight Werewolf Arena's potential as a challenging and scalable LLM benchmark.
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Optimal Strategy in Werewolf Game: A Game Theoretic Perspective
Werewolf game, also known as Mafia game, is a social deduction game that models the conflict between an informed minority (werewolf group) and an uninformed majority (citizen group). This paper explores the optimal strat…
Werewolf: A Straightforward Game Framework with TTS for Improved User Engagement
The growing popularity of social deduction game systems for both business applications and AI research has greatly benefited from the rapid advancements in Large Language Models (LLMs), which now demonstrate stronger rea…
text-to-speechText to SpeechBeyond Survival: Evaluating LLMs in Social Deduction Games with Human-Aligned Strategies
Social deduction games like Werewolf combine language, reasoning, and strategy, providing a testbed for studying natural language and social intelligence. However, most studies reduce the game to LLM-based self-play, yie…
Exploring Large Language Models for Communication Games: An Empirical Study on Werewolf
Communication games, which we refer to as incomplete information games that heavily depend on natural language communication, hold significant research value in fields such as economics, social science, and artificial in…
RetrievalMultiMind: Enhancing Werewolf Agents with Multimodal Reasoning and Theory of Mind
Large Language Model (LLM) agents have demonstrated impressive capabilities in social deduction games (SDGs) like Werewolf, where strategic reasoning and social deception are essential. However, current approaches remain…
Large Language ModelMultimodal Reasoning