Empowering Small Language Models with Factual Hallucination-Aware Reasoning for Financial Classification

2026-01-04 · Han Yuan, Yilin Wu, Li Zhang, Zheng Ma arxiv

Small language models (SLMs) are increasingly used for financial classification due to their fast inference and local deployability. However, compared with large language models, SLMs are more prone to factual hallucinations in reasoning and exhibit weaker classification performance. This raises a natural question: Can mitigating factual hallucinations improve SLMs' financial classification? To address this, we propose a three-step pipeline named AAAI (Association Identification, Automated Detection, and Adaptive Inference). Experiments on three representative SLMs reveal that: (1) factual hallucinations are positively correlated with misclassifications; (2) encoder-based verifiers effectively detect factual hallucinations; and (3) incorporating feedback on factual errors enables SLMs' adaptive inference that enhances classification performance. We hope this pipeline contributes to trustworthy and effective applications of SLMs in finance.

📄 PDF Abstract BibTeX arXiv:2601.01378

Code (0)

등록된 구현이 없습니다.

MACD: Model-Aware Contrastive Decoding via Counterfactual Data

2026-02-02 · Qixin Xiao, Kun Zhou arxiv

Video language models (Video-LLMs) are prone to hallucinations, generating plausible but ungrounded content when visual evidence is weak, ambiguous, or biased. Existing methods, such as contrastive decoding (CD), rely on…

FLAME: Factuality-Aware Alignment for Large Language Models

2024-05-02 · Sheng-Chieh Lin, Luyu Gao, Barlas Oguz, Wenhan Xiong 외

Alignment is a standard procedure to fine-tune pre-trained large language models (LLMs) to follow natural language instructions and serve as helpful AI assistants. We have observed, however, that the conventional alignme…

Hallucination Instruction Following Reinforcement Learning (RL)

Geometry-Aware Hallucination Detection in Large Language Models

2026-01-08 · Bodla Krishna Vamshi, Rohan Bhatnagar, Haizhao Yang arxiv

Large language models (LLMs) frequently generate factually incorrect or unsupported content, commonly referred to as hallucinations. Prior work has explored decoding strategies, retrieval augmentation, and supervised fin…

Hallucination Augmented Recitations for Language Models

2023-11-13 · Abdullatif Köksal, Renat Aksitov, Chung-Ching Chang

Attribution is a key concept in large language models (LLMs) as it enables control over information sources and enhances the factuality of LLMs. While existing approaches utilize open book question answering to improve a…

counterfactual Hallucination Question Answering

Learning to Trust Your Feelings: Leveraging Self-awareness in LLMs for Hallucination Mitigation

2024-01-27 · Yuxin Liang, Zhuoyang Song, Hao Wang, Jiaxing Zhang

We evaluate the ability of Large Language Models (LLMs) to discern and express their internal knowledge state, a key factor in countering factual hallucination and ensuring reliable application of LLMs. We observe a robu…

Hallucination Knowledge Probing reinforcement-learning Reinforcement Learning