QRFA: A Data-Driven Model of Information-Seeking Dialogues
Understanding the structure of interaction processes helps us to improve information-seeking dialogue systems. Analyzing an interaction process boils down to discovering patterns in sequences of alternating utterances exchanged between a user and an agent. Process mining techniques have been successfully applied to analyze structured event logs, discovering the underlying process models or evaluating whether the observed behavior is in conformance with the known process. In this paper, we apply process mining techniques to discover patterns in conversational transcripts and extract a new model of information-seeking dialogues, QRFA, for Query, Request, Feedback, Answer. Our results are grounded in an empirical evaluation across multiple conversational datasets from different domains, which was never attempted before. We show that the QRFA model better reflects conversation flows observed in real information-seeking conversations than models proposed previously. Moreover, QRFA allows us to identify malfunctioning in dialogue system transcripts as deviations from the expected conversation flow described by the model via conformance analysis.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Grounded and Transparent Response Generation for Conversational Information-Seeking Systems
While previous conversational information-seeking (CIS) research has focused on passage retrieval, reranking, and query rewriting, the challenge of synthesizing retrieved information into coherent responses remains. The …
Conversational Response GenerationPassage RetrievalRerankingResponse Generation+1A Large-Scale Analysis of Mixed Initiative in Information-Seeking Dialogues for Conversational Search
Conversational search is a relatively young area of research that aims at automating an information-seeking dialogue. In this paper we help to position it with respect to other research areas within conversational Artifi…
Conversational SearchTowards Asking Clarification Questions for Information Seeking on Task-Oriented Dialogues
Task-oriented dialogue systems aim at providing users with task-specific services. Users of such systems often do not know all the information about the task they are trying to accomplish, requiring them to seek informat…
Question GenerationQuestion-GenerationTask-Oriented Dialogue SystemsEvaluating Large Language Models for Document-grounded Response Generation in Information-Seeking Dialogues
In this paper, we investigate the use of large language models (LLMs) like ChatGPT for document-grounded response generation in the context of information-seeking dialogues. For evaluation, we use the MultiDoc2Dial corpu…
Response GenerationIntroducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset
Conversational search is an approach to information retrieval (IR), where users engage in a dialogue with an agent in order to satisfy their information needs. Previous conceptual work described properties and actions a …
Conversational SearchInformation RetrievalRetrieval