Enhancing Hallucination Detection through Noise Injection
Large Language Models (LLMs) are prone to generating plausible yet incorrect responses, known as hallucinations. Effectively detecting hallucinations is therefore crucial for the safe deployment of LLMs. Recent research has linked hallucinations to model uncertainty, suggesting that hallucinations can be detected by measuring dispersion over answer distributions obtained from a set of samples drawn from a model. While drawing from the distribution over tokens defined by the model is a natural way to obtain samples, in this work, we argue that it is sub-optimal for the purpose of detecting hallucinations. We show that detection can be improved significantly by taking into account model uncertainty in the Bayesian sense. To this end, we propose a very simple and efficient approach that perturbs an appropriate subset of model parameters, or equivalently hidden unit activations, during sampling. We demonstrate its effectiveness across a wide range of datasets and model architectures.
Code (0)
등록된 구현이 없습니다.
Tasks
HallucinationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Prompt Injection Detection and Mitigation via AI Multi-Agent NLP Frameworks
Prompt injection constitutes a significant challenge for generative AI systems by inducing unintended outputs. We introduce a multi-agent NLP framework specifically designed to address prompt injection vulnerabilities th…
HallucinationNoise Augmented Fine Tuning for Mitigating Hallucinations in Large Language Models
Large language models (LLMs) often produce inaccurate or misleading content-hallucinations. To address this challenge, we introduce Noise-Augmented Fine-Tuning (NoiseFiT), a novel framework that leverages adaptive noise …
HallucinationLanguage ModelingLanguage ModellingWhat are Models Thinking about? Understanding Large Language Model Hallucinations "Psychology" through Model Inner State Analysis
Large language model (LLM) systems suffer from the models' unstable ability to generate valid and factual content, resulting in hallucination generation. Current hallucination detection methods heavily rely on out-of-mod…
HallucinationLanguage ModelingLanguage ModellingLarge Language Model+3Hallucinations in Neural Automatic Speech Recognition: Identifying Errors and Hallucinatory Models
Hallucinations are a type of output error produced by deep neural networks. While this has been studied in natural language processing, they have not been researched previously in automatic speech recognition. Here, we d…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Hallucinationspeech-recognition+1CausalGaze: Unveiling Hallucinations via Counterfactual Graph Intervention in Large Language Models
Despite the groundbreaking advancements made by large language models (LLMs), hallucination remains a critical bottleneck for their deployment in high-stakes domains. Existing classification-based methods mainly rely on …