Fact Grounded Attention: Eliminating Hallucination in Large Language Models Through Attention Level Knowledge Integration
"The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge." Large Language Models have conquered natural language but remain prisoners of their own probabilistic nature--confidently hallucinating facts they never truly knew. We present Fact Grounded Attention (FGA), a novel architectural modification that transforms unreliable language models into deterministic truth tellers by injecting verifiable knowledge directly into the attention mechanism. Unlike existing approaches that patch hallucinations after generation or prepend retrieved text, FGA intervenes at the mathematical heart of the transformer--the pre-softmax attention scores--creating a model that cannot hallucinate when facts exist in its knowledge base. Our experiments across 1,107 technical queries spanning smartphones, laptops, and electric vehicles demonstrate a transformation from 6.3% accuracy in vanilla Llama 3.2 to 99.7% accuracy with FGA. More critically, knowledge updates occur in under one second without retraining, compared to hours for parameter editing approaches. FGA doesn't just reduce hallucination--it eliminates it entirely for verifiable facts, marking a fundamental shift from probabilistic approximation to deterministic precision in neural language generation.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Attention Sinks as Internal Signals for Hallucination Detection in Large Language Models
Large language models frequently exhibit hallucinations: fluent and confident outputs that are factually incorrect or unsupported by the input context. While recent hallucination detection methods have explored various f…
Attention Head Embeddings with Trainable Deep Kernels for Hallucination Detection in LLMs
We present a novel approach for detecting hallucinations in large language models (LLMs) by analyzing the probabilistic divergence between prompt and response hidden-state distributions. Counterintuitively, we find that …
HallucinationCausally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs
Object hallucination remains a critical challenge in Large Vision-Language Models (LVLMs), where models generate content inconsistent with visual inputs. Existing language-decoder based mitigation approaches often regula…
Visual GroundingSAVAA: Mitigating Hallucinations in LVLMs via Step-wise Adaptive Visual Attention Amplification
A line of recent training-free methods for mitigating hallucinations in large vision-language models (LVLMs) operates by amplifying attention to visual tokens during autoregressive generation within a single forward pass…
Visual GroundingA Cause-Effect Look at Alleviating Hallucination of Knowledge-grounded Dialogue Generation
Empowered by the large-scale pretrained language models, existing dialogue systems have demonstrated impressive performance conducting fluent and natural-sounding conversations. However, they are still plagued by the hal…
counterfactualCounterfactual ReasoningDialogue GenerationHallucination