Representative Image Feature Extraction via Contrastive Learning Pretraining for Chest X-ray Report Generation
Medical report generation is a challenging task since it is time-consuming and requires expertise from experienced radiologists. The goal of medical report generation is to accurately capture and describe the image findings. Previous works pretrain their visual encoding neural networks with large datasets in different domains, which cannot learn general visual representation in the specific medical domain. In this work, we propose a medical report generation framework that uses a contrastive learning approach to pretrain the visual encoder and requires no additional meta information. In addition, we adopt lung segmentation as an augmentation method in the contrastive learning framework. This segmentation guides the network to focus on encoding the visual feature within the lung region. Experimental results show that the proposed framework improves the performance and the quality of the generated medical reports both quantitatively and qualitatively.
Code (0)
등록된 구현이 없습니다.
Tasks
Contrastive LearningMedical Report GenerationSegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Adversarial Pretraining of Self-Supervised Deep Networks: Past, Present and Future
In this paper, we review adversarial pretraining of self-supervised deep networks including both convolutional neural networks and vision transformers. Unlike the adversarial training with access to labeled examples, adv…
Contrastive LearningMiscellaneousUnsupervised Feature Learning by Autoencoder and Prototypical Contrastive Learning for Hyperspectral Classification
Unsupervised learning methods for feature extraction are becoming more and more popular. We combine the popular contrastive learning method (prototypical contrastive learning) and the classic representation learning meth…
Contrastive LearningGeneral ClassificationRepresentation LearningTissue-Contrastive Semi-Masked Autoencoders for Segmentation Pretraining on Chest CT
Existing Masked Image Modeling (MIM) depends on a spatial patch-based masking-reconstruction strategy to perceive objects'features from unlabeled images, which may face two limitations when applied to chest CT: 1) ineffi…
Contrastive LearningSelf-Supervised LearningTransfer LearningUnderstanding Self-Supervised Pretraining with Part-Aware Representation Learning
In this paper, we are interested in understanding self-supervised pretraining through studying the capability that self-supervised representation pretraining methods learn part-aware representations. The study is mainly …
Contrastive LearningObjectRepresentation LearningUSCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learning
Most deep neural networks (DNNs) based ultrasound (US) medical image analysis models use pretrained backbones (e.g., ImageNet) for better model generalization. However, the domain gap between natural and medical images c…
Contrastive LearningMedical Image AnalysisRepresentation Learning