Finding-Aware Anatomical Tokens for Chest X-Ray Automated Reporting
The task of radiology reporting comprises describing and interpreting the medical findings in radiographic images, including description of their location and appearance. Automated approaches to radiology reporting require the image to be encoded into a suitable token representation for input to the language model. Previous methods commonly use convolutional neural networks to encode an image into a series of image-level feature map representations. However, the generated reports often exhibit realistic style but imperfect accuracy. Inspired by recent works for image captioning in the general domain in which each visual token corresponds to an object detected in an image, we investigate whether using local tokens corresponding to anatomical structures can improve the quality of the generated reports. We introduce a novel adaptation of Faster R-CNN in which finding detection is performed for the candidate bounding boxes extracted during anatomical structure localisation. We use the resulting bounding box feature representations as our set of finding-aware anatomical tokens. This encourages the extracted anatomical tokens to be informative about the findings they contain (required for the final task of radiology reporting). Evaluating on the MIMIC-CXR dataset of chest X-Ray images, we show that task-aware anatomical tokens give state-of-the-art performance when integrated into an automated reporting pipeline, yielding generated reports with improved clinical accuracy.
Code (0)
등록된 구현이 없습니다.
Tasks
Image CaptioningLanguage ModellingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
GRCD: Grounded Region Change Detection for Multi-Finding Chest X-Ray Pairs
Radiologists routinely compare current and prior chest X-rays to track disease progression, producing follow-up reports that describe multiple findings, each localised to an anatomical region and annotated with a tempora…
Change DetectionText GenerationChest X-ray Report Generation through Fine-Grained Label Learning
Obtaining automated preliminary read reports for common exams such as chest X-rays will expedite clinical workflows and improve operational efficiencies in hospitals. However, the quality of reports generated by current …
AnaXNet: Anatomy Aware Multi-label Finding Classification in Chest X-ray
Radiologists usually observe anatomical regions of chest X-ray images as well as the overall image before making a decision. However, most existing deep learning models only look at the entire X-ray image for classificat…
AnatomyClassificationimage-classificationImage Classification+1CheXtriev: Anatomy-Centered Representation for Case-Based Retrieval of Chest Radiographs
We present CheXtriev, a graph-based, anatomy-aware framework for chest radiograph retrieval. Unlike prior methods focussed on global features, our method leverages graph transformers to extract informative features from …
Spatially-Preserving Flattening for Location-Aware Classification of Findings in Chest X-Rays
Chest X-rays have become the focus of vigorous deep learning research in recent years due to the availability of large labeled datasets. While classification of anomalous findings is now possible, ensuring that they are …
Anomaly ClassificationClassificationDeep Learning