Improving Medical Visual Representations via Radiology Report Generation
Vision-language pretraining has been shown to produce high-quality visual encoders which transfer efficiently to downstream computer vision tasks. Contrastive learning approaches have increasingly been adopted for medical vision language pretraining (MVLP), yet recent developments in generative AI offer new modeling alternatives. This paper introduces RadTex, a CNN-encoder transformer-decoder architecture optimized for radiology. We explore bidirectional captioning as an alternative MVLP strategy and demonstrate that RadTex's captioning pretraining is competitive with established contrastive methods, achieving a CheXpert macro-AUC of 89.4%. Additionally, RadTex's lightweight text decoder not only generates clinically relevant radiology reports (macro-F1 score of 0.349), but also provides targeted, interactive responses, highlighting the utility of bidirectional captioning in advancing medical image analysis.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningDecoderImage CaptioningMedical Image AnalysisSimilar Papers 제목 키워드 기반
Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generation
Radiology Report Generation (RRG) is a critical step toward automating healthcare workflows, facilitating accurate patient assessments, and reducing the workload of medical professionals. Despite recent progress in Large…
Medical Report GenerationVisual ReasoningLearning Semi-Structured Representations of Radiology Reports
Beyond their primary diagnostic purpose, radiology reports have been an invaluable source of information in medical research. Given a corpus of radiology reports, researchers are often interested in identifying a subset …
DiagnosticDesigning a Robust Radiology Report Generation System
Recent advances in deep learning have enabled researchers to explore tasks at the intersection of computer vision and natural language processing, such as image captioning, visual question answering, visual dialogue, and…
Decision MakingDiagnosticDiversityImage Captioning+2MCA-RG: Enhancing LLMs with Medical Concept Alignment for Radiology Report Generation
Despite significant advancements in adapting Large Language Models (LLMs) for radiology report generation (RRG), clinical adoption remains challenging due to difficulties in accurately mapping pathological and anatomical…
Contrastive LearningMedical Report Generation based on Segment-Enhanced Contrastive Representation Learning
Automated radiology report generation has the potential to improve radiology reporting and alleviate the workload of radiologists. However, the medical report generation task poses unique challenges due to the limited av…
Contrastive LearningImage SegmentationMedical Image SegmentationMedical Report Generation+2