RadJEPA: Radiology Encoder for Chest X-Rays via Joint Embedding Predictive Architecture

2026-01-22 · Anas Anwarul Haq Khan, Mariam Husain, Pratik Jalan, Kshitij Jadhav arxiv

Vision-language pretraining has driven progress in medical image representation learning, but it depends on paired image-text data and can inherit reporting bias from clinical narratives. We study whether language-free predictive pretraining can produce an image encoder that transfers effectively to radiology report generation. RadJEPA is a chest-X-ray adaptation of I-JEPA, pretrained on approximately 840K unlabeled radiographs using latent context-to-target prediction. Our primary contribution is an extensive empirical evaluation of this language-free encoder for report generation: the frozen image encoder is coupled to a trainable two-layer projector and language decoder, and is also substituted into four established vision-language backbones. Across MIMIC-CXR and IU-Xray, RadJEPA matches or exceeds the evaluated image-only and image-text baselines on lexical, entity-relation, and clinical-label metrics. Controlled MIMIC-only comparisons provide evidence that the predictive objective contributes beyond domain-specific pretraining, while broader comparisons also reflect differences in pretraining data, model capacity, and input resolution. Complementary classification and segmentation experiments assess transfer beyond report generation.

📄 PDF Abstract BibTeX arXiv:2601.15891

Code (0)

등록된 구현이 없습니다.

Tasks

Representation Learning Semantic Segmentation

Gla-AI4BioMed at RRG24: Visual Instruction-tuned Adaptation for Radiology Report Generation

2024-12-06 · Xi Zhang, Zaiqiao Meng, Jake Lever, Edmond S. L. Ho

We introduce a radiology-focused visual language model designed to generate radiology reports from chest X-rays. Building on previous findings that large language models (LLMs) can acquire multimodal capabilities when al…

Language Modeling Language Modelling

Improving Joint Learning of Chest X-Ray and Radiology Report by Word Region Alignment

2021-09-04 · Zhanghexuan Ji, Mohammad Abuzar Shaikh, Dana Moukheiber, Sargur Srihari 외

Self-supervised learning provides an opportunity to explore unlabeled chest X-rays and their associated free-text reports accumulated in clinical routine without manual supervision. This paper proposes a Joint Image Text…

Representation Learning Self-Supervised Learning Sentence Triplet

Effect of Radiology Report Labeler Quality on Deep Learning Models for Chest X-Ray Interpretation

2021-04-01 · Saahil Jain, Akshay Smit, Andrew Y. Ng, Pranav Rajpurkar

Although deep learning models for chest X-ray interpretation are commonly trained on labels generated by automatic radiology report labelers, the impact of improvements in report labeling on the performance of chest X-ra…

Classification General Classification image-classification Image Classification+1

Learning Visual-Semantic Embeddings for Reporting Abnormal Findings on Chest X-rays

2020-10-06 · Findings of the Association for Computational Linguistics 2020 · Jianmo Ni, Chun-Nan Hsu, Amilcare Gentili, Julian McAuley

Automatic medical image report generation has drawn growing attention due to its potential to alleviate radiologists' workload. Existing work on report generation often trains encoder-decoder networks to generate complet…

Clustering Cross-Modal Retrieval Decoder Retrieval+1

TRACE: Temporal Radiology with Anatomical Change Explanation for Grounded X-ray Report Generation

2026-02-03 · OFM Riaz Rahman Aranya, Kevin Desai arxiv

Temporal comparison of chest X-rays is fundamental to clinical radiology, enabling detection of disease progression, treatment response, and new findings. While vision-language models have advanced single-image report ge…

Change Detection Visual Grounding