Characterizing Language Use in a Collaborative Situated Game
Cooperative video games, where multiple participants must coordinate by communicating and reasoning under uncertainty in complex environments, yield a rich source of language data. We collect the Portal Dialogue Corpus: a corpus of 11.5 hours of spoken human dialogue in the co-op mode of the popular Portal 2 virtual puzzle game, comprising 24.5K total utterances. We analyze player language and behavior, identifying a number of linguistic phenomena that rarely appear in most existing chitchat or task-oriented dialogue corpora, including complex spatial reference, clarification and repair, and ad-hoc convention formation. To support future analyses of language use in complex, situated, collaborative problem-solving scenarios, we publicly release the corpus, which comprises player videos, audio, transcripts, game state data, and both manual and automatic annotations of language data.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Executing Instructions in Situated Collaborative Interactions
We study a collaborative scenario where a user not only instructs a system to complete tasks, but also acts alongside it. This allows the user to adapt to the system abilities by changing their language or deciding to si…
Pow-Wow: A Dataset and Study on Collaborative Communication in Pommerman
In multi-agent learning, agents must coordinate with each other in order to succeed. For humans, this coordination is typically accomplished through the use of language. In this work we perform a controlled study of huma…
MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks
An ideal integration of autonomous agents in a human world implies that they are able to collaborate on human terms. In particular, theory of mind plays an important role in maintaining common ground during human collabo…
MinecraftTheory of Mind ModelingCollecting Visually-Grounded Dialogue with A Game Of Sorts
An idealized, though simplistic, view of the referring expression production and grounding process in (situated) dialogue assumes that a speaker must merely appropriately specify their expression so that the target refer…
Coreference ResolutionImage RetrievalReferring ExpressionReferring Expression Comprehension+4SituatedThinker: Grounding LLM Reasoning with Real-World through Situated Thinking
Recent advances in large language models (LLMs) demonstrate their impressive reasoning capabilities. However, the reasoning confined to internal parametric space limits LLMs' access to real-time information and understan…
Mathematical ReasoningMulti-hop Question AnsweringQuestion Answeringtext-based games