Multilingual Hallucination Gaps in Large Language Models
Large language models (LLMs) are increasingly used as alternatives to traditional search engines given their capacity to generate text that resembles human language. However, this shift is concerning, as LLMs often generate hallucinations, misleading or false information that appears highly credible. In this study, we explore the phenomenon of hallucinations across multiple languages in freeform text generation, focusing on what we call multilingual hallucination gaps. These gaps reflect differences in the frequency of hallucinated answers depending on the prompt and language used. To quantify such hallucinations, we used the FactScore metric and extended its framework to a multilingual setting. We conducted experiments using LLMs from the LLaMA, Qwen, and Aya families, generating biographies in 19 languages and comparing the results to Wikipedia pages. Our results reveal variations in hallucination rates, especially between high and low resource languages, raising important questions about LLM multilingual performance and the challenges in evaluating hallucinations in multilingual freeform text generation.
Code (0)
등록된 구현이 없습니다.
Tasks
HallucinationText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Teaching LLMs to Abstain across Languages via Multilingual Feedback
Multilingual LLMs often have knowledge disparities across languages, with larger gaps in under-resourced languages. Teaching LLMs to abstain in the face of knowledge gaps is thus a promising strategy to mitigate hallucin…
Language ModelingLanguage ModellingComparing Hallucination Detection Metrics for Multilingual Generation
While many hallucination detection techniques have been evaluated on English text, their effectiveness in multilingual contexts remains unknown. This paper assesses how well various factual hallucination detection metric…
HallucinationNatural Language InferenceSentenceSUTRA: Scalable Multilingual Language Model Architecture
In this paper, we introduce SUTRA, multilingual Large Language Model architecture capable of understanding, reasoning, and generating text in over 50 languages. SUTRA's design uniquely decouples core conceptual understan…
Computational EfficiencyHallucinationLanguage ModelingLanguage Modelling+4Mitigating Multilingual Hallucination in Large Vision-Language Models
While Large Vision-Language Models (LVLMs) have exhibited remarkable capabilities across a wide range of tasks, they suffer from hallucination problems, where models generate plausible yet incorrect answers given the inp…
HallucinationPoly-FEVER: A Multilingual Fact Verification Benchmark for Hallucination Detection in Large Language Models
Hallucinations in generative AI, particularly in Large Language Models (LLMs), pose a significant challenge to the reliability of multilingual applications. Existing benchmarks for hallucination detection focus primarily…
Fact CheckingFact VerificationHallucinationMultilingual NLP