Hallucinations and Key Information Extraction in Medical Texts: A Comprehensive Assessment of Open-Source Large Language Models
Clinical summarization is crucial in healthcare as it distills complex medical data into digestible information, enhancing patient understanding and care management. Large language models (LLMs) have shown significant potential in automating and improving the accuracy of such summarizations due to their advanced natural language understanding capabilities. These models are particularly applicable in the context of summarizing medical/clinical texts, where precise and concise information transfer is essential. In this paper, we investigate the effectiveness of open-source LLMs in extracting key events from discharge reports, such as reasons for hospital admission, significant in-hospital events, and critical follow-up actions. In addition, we also assess the prevalence of various types of hallucinations in the summaries produced by these models. Detecting hallucinations is vital as it directly influences the reliability of the information, potentially affecting patient care and treatment outcomes. We conduct comprehensive numerical simulations to rigorously evaluate the performance of these models, further probing the accuracy and fidelity of the extracted content in clinical summarization.
Code (0)
등록된 구현이 없습니다.
Tasks
Key Information ExtractionNatural Language UnderstandingSimilar Papers 제목 키워드 기반
BiomedRAG: A Retrieval Augmented Large Language Model for Biomedicine
Large Language Models (LLMs) have swiftly emerged as vital resources for different applications in the biomedical and healthcare domains; however, these models encounter issues such as generating inaccurate information o…
Language ModelingLanguage ModellingLarge Language ModelLink Prediction+6MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models
Medical Large Language Models (MLLMs) have demonstrated potential in healthcare applications, yet their propensity for hallucinations -- generating medically implausible or inaccurate information -- presents substantial …
Hallucinationreinforcement-learningReinforcement LearningMitigating Hallucinations of Large Language Models in Medical Information Extraction via Contrastive Decoding
The impressive capabilities of large language models (LLMs) have attracted extensive interests of applying LLMs to medical field. However, the complex nature of clinical environments presents significant hallucination ch…
HallucinationSupporting Medical Relation Extraction via Causality-Pruned Semantic Dependency Forest
Medical Relation Extraction (MRE) task aims to extract relations between entities in medical texts. Traditional relation extraction methods achieve impressive success by exploring the syntactic information, e.g., depende…
Medical Relation ExtractionRelationRelation ExtractionThe Effects of Hallucinations in Synthetic Training Data for Relation Extraction
Relation extraction is crucial for constructing knowledge graphs, with large high-quality datasets serving as the foundation for training, fine-tuning, and evaluating models. Generative data augmentation (GDA) is a commo…
Data AugmentationKnowledge GraphsRelationRelation Extraction+1