Views Are My Own, but Also Yours: Benchmarking Theory of Mind Using Common Ground
Evaluating the theory of mind (ToM) capabilities of language models (LMs) has recently received a great deal of attention. However, many existing benchmarks rely on synthetic data, which risks misaligning the resulting experiments with human behavior. We introduce the first ToM dataset based on naturally occurring spoken dialogs, Common-ToM, and show that LMs struggle to demonstrate ToM. We then show that integrating a simple, explicit representation of beliefs improves LM performance on Common-ToM.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingSimilar Papers 제목 키워드 기반
A formal definition and meta-model for a machine theory of mind
This paper proposes, for the first time, a rigorous formal definition of the concept of Machine Theory of Mind, based on principles supported by evidence from cognitive psychology, neuroscience and artificial intelligenc…
Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning
Theory of Mind (ToM) capabilities in LLMs have recently become a central object of investigation. Cognitive science distinguishes between two steps required for ToM tasks: 1) determine whether to invoke ToM, which includ…
BenchmarkingPositionMind the Motions: Benchmarking Theory-of-Mind in Everyday Body Language
Our ability to interpret others' mental states through nonverbal cues (NVCs) is fundamental to our survival and social cohesion. While existing Theory of Mind (ToM) benchmarks have primarily focused on false-belief tasks…
Minds, Brains, AI
In the last year or so and going back many decades there has been extensive claims by major computational scientists, engineers, and others that AGI, artificial general intelligence, is five or ten years away, but withou…
Self-Driving CarsGroupToM-Bench: Benchmarking Group Theory of Mind and Nonlinear Social Emergence in MLLMs
True general intelligence requires not only a model of the physical world but also a social world model: the capacity to infer how individual mental states interact and crystallize into group-level outcomes. Despite nota…