paper-with-me

홈 › Papers

Collaborative Quest Completion with LLM-driven Non-Player Characters in Minecraft

2024-07-03 · Sudha Rao, Weijia Xu, Michael Xu, Jorge Leandro, Ken Lobb, Gabriel DesGarennes, Chris Brockett, Bill Dolan

The use of generative AI in video game development is on the rise, and as the conversational and other capabilities of large language models continue to improve, we expect LLM-driven non-player characters (NPCs) to become widely deployed. In this paper, we seek to understand how human players collaborate with LLM-driven NPCs to accomplish in-game goals. We design a minigame within Minecraft where a player works with two GPT4-driven NPCs to complete a quest. We perform a user study in which 28 Minecraft players play this minigame and share their feedback. On analyzing the game logs and recordings, we find that several patterns of collaborative behavior emerge from the NPCs and the human players. We also report on the current limitations of language-only models that do not have rich game-state or visual understanding. We believe that this preliminary study and analysis will inform future game developers on how to better exploit these rapidly improving generative AI models for collaborative roles in games.

📄 PDF Abstract BibTeX arXiv:2407.03460

Code (0)

등록된 구현이 없습니다.

Tasks

Minecraft

Similar Papers 제목 키워드 기반

Evaluation of the KomParse Conversational Non-Player Characters in a Commercial Virtual World

2012-05-01 · LREC 2012 5 · Tina Kluewer, Feiyu Xu, Peter Adolphs, Hans Uszkoreit

The paper describes the evaluation of the KomParse system. KomParse is a dialogue system embedded in a 3-D massive multiplayer online game, allowing conversations between non player characters (NPCs) and game users. In a…

Natural Language UnderstandingQuestion Answering

Immersive Text Game and Personality Classification

2022-03-20 · Wanshui Li, Yifan Bai, Jiaxuan Lu, Kexin Yi

We designed and built a game called \textit{Immersive Text Game}, which allows the player to choose a story and a character, and interact with other characters in the story in an immersive manner of dialogues. The game i…

ClassificationLanguage ModelingLanguage ModellingText Generation

Instruction Clarification Requests in Multimodal Collaborative Dialogue Games: Tasks, and an Analysis of the CoDraw Dataset

2023-02-28 · Brielen Madureira, David Schlangen

In visual instruction-following dialogue games, players can engage in repair mechanisms in face of an ambiguous or underspecified instruction that cannot be fully mapped to actions in the world. In this work, we annotate…

Instruction Followingvisual instruction following

An Empirical Evaluation of AI-Powered Non-Player Characters' Perceived Realism and Performance in Virtual Reality Environments

2025-07-14 · Mikko Korkiakoski, Saeid Sheikhi, Jesper Nyman, Jussi Saariniemi 외

Advancements in artificial intelligence (AI) have significantly enhanced the realism and interactivity of non-player characters (NPCs) in virtual reality (VR), creating more engaging and believable user experiences. This…

Speech-to-Texttext-to-speechText to Speech

Player-Driven Emergence in LLM-Driven Game Narrative

2024-04-25 · Xiangyu Peng, Jessica Quaye, Sudha Rao, Weijia Xu 외

We explore how interaction with large language models (LLMs) can give rise to emergent behaviors, empowering players to participate in the evolution of game narratives. Our testbed is a text-adventure game in which playe…

Language ModelingLanguage ModellingLarge Language Model