Decision-Theoretic Question Generation for Situated Reference Resolution: An Empirical Study and Computational Model
Dialogue agents that interact with humans in situated environments need to manage referential ambiguity across multiple modalities and ask for help as needed. However, it is not clear what kinds of questions such agents should ask nor how the answers to such questions can be used to resolve ambiguity. To address this, we analyzed dialogue data from an interactive study in which participants controlled a virtual robot tasked with organizing a set of tools while engaging in dialogue with a live, remote experimenter. We discovered a number of novel results, including the distribution of question types used to resolve ambiguity and the influence of dialogue-level factors on the reference resolution process. Based on these empirical findings we: (1) developed a computational model for clarification requests using a decision network with an entropy-based utility assignment method that operates across modalities, (2) evaluated the model, showing that it outperforms a slot-filling baseline in environments of varying ambiguity, and (3) interpreted the results to offer insight into the ways that agents can ask questions to facilitate situated reference resolution.
Code (0)
등록된 구현이 없습니다.
Tasks
Question GenerationQuestion-Generationslot-fillingSlot FillingSimilar Papers 제목 키워드 기반
Situated Natural Language Explanations
Natural language is among the most accessible tools for explaining decisions to humans, and large pretrained language models (PLMs) have demonstrated impressive abilities to generate coherent natural language explanation…
Prompt EngineeringHow Should Agents Ask Questions For Situated Learning? An Annotated Dialogue Corpus
Intelligent agents that are confronted with novel concepts in situated environments will need to ask their human teammates questions to learn about the physical world. To better understand this problem, we need data abou…
Novel ConceptsQuestion GenerationQuestion-GenerationAnnotating anaphoric phenomena in situated dialogue
In recent years several corpora have been developed for vision and language tasks. With this paper, we intend to start a discussion on the annotation of referential phenomena in situated dialogue. We argue that there is …
coreference-resolutionCoreference ResolutionSOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge
Learning commonsense reasoning from visual contexts and scenes in real-world is a crucial step toward advanced artificial intelligence. However, existing video reasoning benchmarks are still inadequate since they were ma…
General KnowledgeKnowledge GraphsWorld KnowledgeA System For Robot Concept Learning Through Situated Dialogue
Robots operating in unexplored environments with human teammates will need to learn unknown concepts on the fly. To this end, we demonstrate a novel system that combines a computational model of question generation with …
One-Shot LearningQuestion GenerationQuestion-Generation