Feature Difference Makes Sense: A medical image captioning model exploiting feature difference and tag information
Medical image captioning can reduce the workload of physicians and save time and expense by automatically generating reports. However, current datasets are small and limited, creating additional challenges for researchers. In this study, we propose a feature difference and tag information combined long short-term memory (LSTM) model for chest x-ray report generation. A feature vector extracted from the image conveys visual information, but its ability to describe the image is limited. Other image captioning studies exhibited improved performance by exploiting feature differences, so the proposed model also utilizes them. First, we propose a difference and tag (DiTag) model containing the difference between the patient and normal images. Then, we propose a multi-difference and tag (mDiTag) model that also contains information about low-level differences, such as contrast, texture, and localized area. Evaluation of the proposed models demonstrates that the mDiTag model provides more information to generate captions and outperforms all other models.
Code (0)
등록된 구현이 없습니다.
Tasks
Image CaptioningTAGSimilar Papers 제목 키워드 기반
What Makes Transfer Learning Work For Medical Images: Feature Reuse & Other Factors
Transfer learning is a standard technique to transfer knowledge from one domain to another. For applications in medical imaging, transfer from ImageNet has become the de-facto approach, despite differences in the tasks a…
Inductive BiasTransfer LearningDAN: A Deformation-Aware Network for Consecutive Biomedical Image Interpolation
The continuity of biological tissue between consecutive biomedical images makes it possible for the video interpolation algorithm, to recover large area defects and tears that are common in biomedical images. However, no…
Evaluating the trustworthiness of the Fréchet Inception Distance with stochastic embedding representations
Feature embeddings acquired from pretrained models are widely used in medical applications of deep learning to assess the characteristics of datasets; e.g. to determine the quality of synthetic, generated medical images.…
WME: Sense, Polarity and Affinity based Concept Resource for Medical Events
In order to overcome the lack of medical corpora, we have developed a WordNet for Medical Events (WME) for identifying medical terms and their sense related information using a seed list. The initial WME resource contain…
POSRelationOn-the-Fly Test-time Adaptation for Medical Image Segmentation
One major problem in deep learning-based solutions for medical imaging is the drop in performance when a model is tested on a data distribution different from the one that it is trained on. Adapting the source model to t…
Image SegmentationMedical Image SegmentationSemantic SegmentationTest-time Adaptation