Automated Histopathology Report Generation via Pyramidal Feature Extraction and the UNI Foundation Model
Generating diagnostic text from histopathology whole slide images (WSIs) is challenging due to the gigapixel scale of the input and the requirement for precise, domain specific language. We propose a hierarchical vision language framework that combines a frozen pathology foundation model with a Transformer decoder for report generation. To make WSI processing tractable, we perform multi resolution pyramidal patch selection (downsampling factors 2^3 to 2^6) and remove background and artifacts using Laplacian variance and HSV based criteria. Patch features are extracted with the UNI Vision Transformer and projected to a 6 layer Transformer decoder that generates diagnostic text via cross attention. To better represent biomedical terminology, we tokenize the output using BioGPT. Finally, we add a retrieval based verification step that compares generated reports with a reference corpus using Sentence BERT embeddings; if a high similarity match is found, the generated report is replaced with the retrieved ground truth reference to improve reliability.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
HARE: an entity and relation centric evaluation framework for histopathology reports
Medical domain automated text generation is an active area of research and development; however, evaluating the clinical quality of generated reports remains a challenge, especially in instances where domain-specific met…
Relation ExtractionText GenerationHistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interaction
Histopathology serves as the gold standard in cancer diagnosis, with clinical reports being vital in interpreting and understanding this process, guiding cancer treatment and patient care. The automation of histopatholog…
DiagnosticMedical Report GenerationMultiple Instance LearningSurvival Analysis+2Leveraging large language models for structured information extraction from pathology reports
Background: Structured information extraction from unstructured histopathology reports facilitates data accessibility for clinical research. Manual extraction by experts is time-consuming and expensive, limiting scalabil…
Automatic Report Generation for Histopathology images using pre-trained Vision Transformers
Deep learning for histopathology has been successfully used for disease classification, image segmentation and more. However, combining image and text modalities using current state-of-the-art methods has been a challeng…
DecoderImage SegmentationSemantic Segmentationzero-shot-classification+1Automatic Report Generation for Histopathology images using pre-trained Vision Transformers and BERT
Deep learning for histopathology has been successfully used for disease classification, image segmentation and more. However, combining image and text modalities using current state-of-the-art (SOTA) methods has been a c…
Caption GenerationDecoderHistopathological Image ClassificationImage Captioning+4