Boosting Healthcare LLMs Through Retrieved Context

2024-09-23 · Jordi Bayarri-Planas, Ashwin Kumar Gururajan, Dario Garcia-Gasulla

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language processing, and yet, their factual inaccuracies and hallucinations limits their application, particularly in critical domains like healthcare. Context retrieval methods, by introducing relevant information as input, have emerged as a crucial approach for enhancing LLM factuality and reliability. This study explores the boundaries of context retrieval methods within the healthcare domain, optimizing their components and benchmarking their performance against open and closed alternatives. Our findings reveal how open LLMs, when augmented with an optimized retrieval system, can achieve performance comparable to the biggest private solutions on established healthcare benchmarks (multiple-choice question answering). Recognizing the lack of realism of including the possible answers within the question (a setup only found in medical exams), and after assessing a strong LLM performance degradation in the absence of those options, we extend the context retrieval system in that direction. In particular, we propose OpenMedPrompt a pipeline that improves the generation of more reliable open-ended answers, moving this technology closer to practical application.

📄 PDF Abstract BibTeX arXiv:2409.15127

Code (1)

hpai-bsc/prompt_engine 공식 구현

Tasks

BenchmarkingMultiple-choiceQuestion Answering Retrieval

SAGE: A Framework of Precise Retrieval for RAG

2025-03-03 · Jintao Zhang, Guoliang Li, Jinyang Su

Retrieval-augmented generation (RAG) has demonstrated significant proficiency in conducting question-answering (QA) tasks within a specified corpus. Nonetheless, numerous failure instances of RAG in QA still exist. These…

Question Answering RAG Retrieval Retrieval-augmented Generation+1

How Retrieved Context Shapes Internal Representations in RAG

2026-02-23 · Samuel Yeh, Sharon Li arxiv

Retrieval-augmented generation (RAG) enhances large language models (LLMs) by conditioning generation on retrieved external documents, but the effect of retrieved context is often non-trivial. In realistic retrieval sett…

Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts?

2024-01-22 · Hexiang Tan, Fei Sun, Wanli Yang, Yuanzhuo Wang 외

While auxiliary information has become a key to enhancing Large Language Models (LLMs), relatively little is known about how LLMs merge these contexts, specifically contexts generated by LLMs and those retrieved from ext…

Misinformation

Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding

2026-04-24 · Weixu Zhang, Fanghua Ye, Qiang Gao, Jian Li 외 arxiv

Large language models (LLMs) often produce content that contradicts or overlooks information provided in the input context, a phenomenon known as faithfulness hallucination. In this paper, we propose Context-Fidelity Boo…

Semantic Similarity Question Answering

Long$^2$RAG: Evaluating Long-Context & Long-Form Retrieval-Augmented Generation with Key Point Recall

2024-10-30 · Zehan Qi, Rongwu Xu, Zhijiang Guo, Cunxiang Wang 외

Retrieval-augmented generation (RAG) is a promising approach to address the limitations of fixed knowledge in large language models (LLMs). However, current benchmarks for evaluating RAG systems suffer from two key defic…

Form RAG Retrieval Retrieval-augmented Generation