Uncovering Knowledge Gaps in Radiology Report Generation Models through Knowledge Graphs
Recent advancements in artificial intelligence have significantly improved the automatic generation of radiology reports. However, existing evaluation methods fail to reveal the models' understanding of radiological images and their capacity to achieve human-level granularity in descriptions. To bridge this gap, we introduce a system, named ReXKG, which extracts structured information from processed reports to construct a comprehensive radiology knowledge graph. We then propose three metrics to evaluate the similarity of nodes (ReXKG-NSC), distribution of edges (ReXKG-AMS), and coverage of subgraphs (ReXKG-SCS) across various knowledge graphs. We conduct an in-depth comparative analysis of AI-generated and human-written radiology reports, assessing the performance of both specialist and generalist models. Our study provides a deeper understanding of the capabilities and limitations of current AI models in radiology report generation, offering valuable insights for improving model performance and clinical applicability.
Code (1)
Tasks
Knowledge GraphsSimilar Papers 제목 키워드 기반
ReXamine-Global: A Framework for Uncovering Inconsistencies in Radiology Report Generation Metrics
Given the rapidly expanding capabilities of generative AI models for radiology, there is a need for robust metrics that can accurately measure the quality of AI-generated radiology reports across diverse hospitals. We de…
Knowledge Matters: Radiology Report Generation with General and Specific Knowledge
Automatic radiology report generation is critical in clinics which can relieve experienced radiologists from the heavy workload and remind inexperienced radiologists of misdiagnosis or missed diagnose. Existing approache…
DecoderGeneral KnowledgeImage CaptioningCan Prompt Learning Benefit Radiology Report Generation?
Radiology report generation aims to automatically provide clinically meaningful descriptions of radiology images such as MRI and X-ray. Although great success has been achieved in natural scene image captioning tasks, ra…
Image CaptioningPrompt EngineeringPrompt LearningA Self-Guided Framework for Radiology Report Generation
Automatic radiology report generation is essential to computer-aided diagnosis. Through the success of image captioning, medical report generation has been achievable. However, the lack of annotated disease labels is sti…
Image CaptioningMedical Report GenerationPrior Knowledge Enhances Radiology Report Generation
Radiology report generation aims to produce computer-aided diagnoses to alleviate the workload of radiologists and has drawn increasing attention recently. However, previous deep learning methods tend to neglect the mutu…