Do We Need Language-Specific Fact-Checking Models? The Case of Chinese
This paper investigates the potential benefits of language-specific fact-checking models, focusing on the case of Chinese. We first demonstrate the limitations of translation-based methods and multilingual large language models (e.g., GPT-4), highlighting the need for language-specific systems. We further propose a Chinese fact-checking system that can better retrieve evidence from a document by incorporating context information. To better analyze token-level biases in different systems, we construct an adversarial dataset based on the CHEF dataset, where each instance has large word overlap with the original one but holds the opposite veracity label. Experimental results on the CHEF dataset and our adversarial dataset show that our proposed method outperforms translation-based methods and multilingual LLMs and is more robust toward biases, while there is still large room for improvement, emphasizing the importance of language-specific fact-checking systems.
Code (0)
등록된 구현이 없습니다.
Tasks
Evidence SelectionFact CheckingMisinformationSentenceSentence ClassificationTranslationSimilar Papers 제목 키워드 기반
Fact-checking information from large language models can decrease headline discernment
Fact checking can be an effective strategy against misinformation, but its implementation at scale is impeded by the overwhelming volume of information online. Recent artificial intelligence (AI) language models have sho…
Fact CheckingLanguage ModellingLarge Language ModelMisinformationUnderstanding Inequality of LLM Fact-Checking over Geographic Regions with Agent and Retrieval models
Fact-checking is a potentially useful application of Large Language Models (LLMs) to combat the growing dissemination of disinformation. However, the performance of LLMs varies across geographic regions. In this paper, w…
Fact CheckingGeneral KnowledgeRAGRetrieval+1Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checking
The pervasiveness of large language models and generative AI in online media has amplified the need for effective automated fact-checking to assist fact-checkers in tackling the increasing volume and sophistication of mi…
Decision MakingFact CheckingMisinformationExplainable Automated Fact-Checking for Public Health Claims
Fact-checking is the task of verifying the veracity of claims by assessing their assertions against credible evidence. The vast majority of fact-checking studies focus exclusively on political claims. Very little researc…
Explanation GenerationFact CheckingMitigating hallucinations in healthcare LLMs with granular fact-checking and domain-specific adaptation
In healthcare, it is essential for any Large Language Model (LLM)-generated output to be reliable and accurate, particularly in cases involving decision-making and patient safety. However, the outputs are often unreliabl…