Automated Report Generation for Lung Cytological Images Using a CNN Vision Classifier and Multiple-Transformer Text Decoders: Preliminary Study
Cytology plays a crucial role in lung cancer diagnosis. Pulmonary cytology involves cell morphological characterization in the specimen and reporting the corresponding findings, which are extremely burdensome tasks. In this study, we propose a report-generation technique for lung cytology images. In total, 71 benign and 135 malignant pulmonary cytology specimens were collected. Patch images were extracted from the captured specimen images, and the findings were assigned to each image as a dataset for report generation. The proposed method consists of a vision model and a text decoder. In the former, a convolutional neural network (CNN) is used to classify a given image as benign or malignant, and the features related to the image are extracted from the intermediate layer. Independent text decoders for benign and malignant cells are prepared for text generation, and the text decoder switches according to the CNN classification results. The text decoder is configured using a Transformer that uses the features obtained from the CNN for report generation. Based on the evaluation results, the sensitivity and specificity were 100% and 96.4%, respectively, for automated benign and malignant case classification, and the saliency map indicated characteristic benign and malignant areas. The grammar and style of the generated texts were confirmed as correct and in better agreement with gold standard compared to existing LLM-based image-captioning methods and single-text-decoder ablation model. These results indicate that the proposed method is useful for pulmonary cytology classification and reporting.
Code (0)
등록된 구현이 없습니다.
Tasks
DecoderImage CaptioningLung Cancer DiagnosisSpecificityText GenerationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Using Few-Shot Learning to Classify Primary Lung Cancer and Other Malignancy with Lung Metastasis in Cytological Imaging via Endobronchial Ultrasound Procedures
This study presents a computer-aided diagnosis (CAD) system to assist early detection of lung metastases during endobronchial ultrasound (EBUS) procedures, significantly reducing follow-up time and enabling timely treatm…
Contrastive LearningFew-Shot LearningImage Classificationparameter-efficient fine-tuning+1Automated Chest X-Ray Report Generator Using Multi-Model Deep Learning Approach
Reading and interpreting chest X-ray images is one of the most radiologist's routines. However, it still can be challenging, even for the most experienced ones. Therefore, we proposed a multi-model deep learning-based au…
Binary ClassificationSentenceSynthetic Lung Nodule 3D Image Generation Using Autoencoders
One of the challenges of using machine learning techniques with medical data is the frequent dearth of source image data on which to train. A representative example is automated lung cancer diagnosis, where nodule images…
BIG-bench Machine LearningImage GenerationLung Cancer DiagnosisMyData: A Comprehensive Database of Mycetoma Tissue Microscopic Images for Histopathological Analysis
Mycetoma is a chronic and neglected inflammatory disease prevalent in tropical and subtropical regions. It can lead to severe disability and social stigma. The disease is classified into two types based on the causative …
Automated Report-Derived Oncology VQA Benchmark for Evaluating Vision-Language Models on 3D Medical Imaging
Evaluating vision-language models (VLMs) on medical images requires benchmarks that are clinically grounded, scalable, and controlled for evaluation confounds. Existing public benchmarks are limited in scale, manually an…