Detecting Response Generation Not Requiring Factual Judgment
With the remarkable development of large language models (LLMs), ensuring the factuality of output has become a challenge. However, having all the contents of the response with given knowledge or facts is not necessarily a good thing in dialogues. This study aimed to achieve both attractiveness and factuality in a dialogue response for which a task was set to predict sentences that do not require factual correctness judgment such as agreeing, or personal opinions/feelings. We created a dataset, dialogue dataset annotated with fact-check-needed label (DDFC), for this task via crowdsourcing, and classification tasks were performed on several models using this dataset. The model with the highest classification accuracy could yield about 88% accurate classification results.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationResponse GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
KnowHalu: Hallucination Detection via Multi-Form Knowledge Based Factual Checking
This paper introduces KnowHalu, a novel approach for detecting hallucinations in text generated by large language models (LLMs), utilizing step-wise reasoning, multi-formulation query, multi-form knowledge for factual ch…
Fact CheckingFormHallucinationRetrievalFRED: Financial Retrieval-Enhanced Detection and Editing of Hallucinations in Language Models
Hallucinations in large language models pose a critical challenge for applications requiring factual reliability, particularly in high-stakes domains such as finance. This work presents an effective approach for detectin…
Text GenerationImproving Model Factuality with Fine-grained Critique-based Evaluator
Factuality evaluation aims to detect factual errors produced by language models (LMs) and hence guide the development of more factual models. Towards this goal, we train a factuality evaluator, FenCE, that provides LM ge…
Data AugmentationLongSumEval: Question-Answering Based Evaluation and Feedback-Driven Refinement for Long Document Summarization
Evaluating long document summaries remains the primary bottleneck in summarization research. Existing metrics correlate weakly with human judgments and produce aggregate scores without explaining deficiencies or guiding …
Document SummarizationText GenerationFact or Guesswork? Evaluating Large Language Model's Medical Knowledge with Structured One-Hop Judgment
Large language models (LLMs) have been widely adopted in various downstream task domains. However, their ability to directly recall and apply factual medical knowledge remains under-explored. Most existing medical QA ben…
Knowledge GraphsRetrieval-augmented Generation