Can We Catch the Elephant? A Survey of the Evolvement of Hallucination Evaluation on Natural Language Generation
Hallucination in Natural Language Generation (NLG) is like the elephant in the room, obvious but often overlooked until recent achievements significantly improved the fluency and grammaticality of generated text. As the capabilities of text generation models have improved, researchers have begun to pay more attention to the phenomenon of hallucination. Despite significant progress in this field in recent years, the evaluation system for hallucination is complex and diverse, lacking clear organization. We are the first to comprehensively survey how various evaluation methods have evolved with the development of text generation models from three dimensions, including hallucinated fact granularity, evaluator design principles, and assessment facets. This survey aims to help researchers identify current limitations in hallucination evaluation and highlight future research directions.
Code (0)
등록된 구현이 없습니다.
Tasks
HallucinationHallucination EvaluationSurveyText GenerationSimilar Papers 제목 키워드 기반
Real-Time Evaluation Models for RAG: Who Detects Hallucinations Best?
This article surveys Evaluation models to automatically detect hallucinations in Retrieval-Augmented Generation (RAG), and presents a comprehensive benchmark of their performance across six RAG applications. Methods incl…
HallucinationHallucination EvaluationLanguage ModelingLanguage Modelling+3CATCH: Complementary Adaptive Token-level Contrastive Decoding to Mitigate Hallucinations in LVLMs
Large Vision-Language Model (LVLM) systems have demonstrated impressive vision-language reasoning capabilities but suffer from pervasive and severe hallucination issues, posing significant risks in critical domains such …
HallucinationLanguage ModelingLanguage ModellingQuestion Answering+1Poaching Hotspot Identification Using Satellite Imagery
Elephant Poaching in African countries has been a decade-old problem. So much so that African Forest Elephants are now listed as an endangered species, and African Savannah Elephants as critically endangered by the IUCN …
A Survey on Large Language Model Hallucination via a Creativity Perspective
Hallucinations in large language models (LLMs) are always seen as limitations. However, could they also be a source of creativity? This survey explores this possibility, suggesting that hallucinations may contribute to L…
HallucinationLanguage ModelingLanguage ModellingLarge Language Model+1Automatic Detection and Compression for Passive Acoustic Monitoring of the African Forest Elephant
In this work, we consider applying machine learning to the analysis and compression of audio signals in the context of monitoring elephants in sub-Saharan Africa. Earth's biodiversity is increasingly under threat by sour…
Audio Compression