Conformal Intent Classification and Clarification for Fast and Accurate Intent Recognition
We present Conformal Intent Classification and Clarification (CICC), a framework for fast and accurate intent classification for task-oriented dialogue systems. The framework turns heuristic uncertainty scores of any intent classifier into a clarification question that is guaranteed to contain the true intent at a pre-defined confidence level. By disambiguating between a small number of likely intents, the user query can be resolved quickly and accurately. Additionally, we propose to augment the framework for out-of-scope detection. In a comparative evaluation using seven intent recognition datasets we find that CICC generates small clarification questions and is capable of out-of-scope detection. CICC can help practitioners and researchers substantially in improving the user experience of dialogue agents with specific clarification questions.
Code (0)
등록된 구현이 없습니다.
Tasks
intent-classificationIntent ClassificationIntent RecognitionTask-Oriented Dialogue SystemsSimilar Papers 제목 키워드 기반
Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs
Resolving ambiguities through interaction is a hallmark of natural language, and modeling this behavior is a core challenge in crafting AI assistants. In this work, we study such behavior in LMs by proposing a task-agnos…
Machine TranslationNatural Language InferenceQuestion AnsweringPrism: Towards Lowering User Cognitive Load in LLMs via Complex Intent Understanding
Large Language Models are rapidly emerging as web-native interfaces to social platforms. On the social web, users frequently have ambiguous and dynamic goals, making complex intent understanding-rather than single-turn e…
Resolving Intent Ambiguities by Retrieving Discriminative Clarifying Questions
Task oriented Dialogue Systems generally employ intent detection systems in order to map user queries to a set of pre-defined intents. However, user queries appearing in natural language can be easily ambiguous and hence…
Intent DetectionQuestion GenerationQuestion-GenerationTask-Oriented Dialogue SystemsCuriosity by Design: An LLM-based Coding Assistant Asking Clarification Questions
Large Language Models (LLMs) are increasingly used as coding assistants. However, the ambiguity of the developer's prompt often leads to incorrect code generation, as current models struggle to infer user intent without …
Prompt EngineeringCode GenerationClarification Is Not Enough: Post-Clarification Answering Remains the Bottleneck in Multi-Turn QA
Pluralistic alignment requires systems to adapt to diverse user values, communication styles, and contextual assumptions. We believe that a foundational prerequisite for such alignment enabling accurate preference elicit…
Question Answering