Aligning Knowledge Concepts to Whole Slide Images for Precise Histopathology Image Analysis
Due to the large size and lack of fine-grained annotation, Whole Slide Images (WSIs) analysis is commonly approached as a Multiple Instance Learning (MIL) problem. However, previous studies only learn from training data, posing a stark contrast to how human clinicians teach each other and reason about histopathologic entities and factors. Here we present a novel knowledge concept-based MIL framework, named ConcepPath to fill this gap. Specifically, ConcepPath utilizes GPT-4 to induce reliable diseasespecific human expert concepts from medical literature, and incorporate them with a group of purely learnable concepts to extract complementary knowledge from training data. In ConcepPath, WSIs are aligned to these linguistic knowledge concepts by utilizing pathology vision-language model as the basic building component. In the application of lung cancer subtyping, breast cancer HER2 scoring, and gastric cancer immunotherapy-sensitive subtyping task, ConcepPath significantly outperformed previous SOTA methods which lack the guidance of human expert knowledge.
Code (1)
Tasks
Language ModelingLanguage ModellingMultiple Instance Learningwhole slide imagesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
RankMix: Data Augmentation for Weakly Supervised Learning of Classifying Whole Slide Images With Diverse Sizes and Imbalanced Categories
Whole Slide Images (WSIs) are usually gigapixel in size and lack pixel-level annotations. The WSI datasets are also imbalanced in categories. These unique characteristics, significantly different from the ones in nat…
Data AugmentationWeakly-supervised Learningwhole slide imagesSlide-Level Prompt Learning with Vision Language Models for Few-Shot Multiple Instance Learning in Histopathology
In this paper, we address the challenge of few-shot classification in histopathology whole slide images (WSIs) by utilizing foundational vision-language models (VLMs) and slide-level prompt learning. Given the gigapixel …
Multiple Instance LearningPrompt Learningwhole slide imagesNuclei-Location Based Point Set Registration of Multi-Stained Whole Slide Images
Whole Slide Images (WSIs) provide exceptional detail for studying tissue architecture at the cell level. To study tumour microenvironment (TME) with the context of various protein biomarkers and cell sub-types, analysis …
whole slide imagesA Multimodal Knowledge-enhanced Whole-slide Pathology Foundation Model
Remarkable strides in computational pathology have been made in the task-agnostic foundation model that advances the performance of a wide array of downstream clinical tasks. Despite the promising performance, there are …
Diagnosticwhole slide imagesWeakly supervised training of pixel resolution segmentation models on whole slide images
We present a novel approach to train pixel resolution segmentation models on whole slide images in a weakly supervised setup. The model is trained to classify patches extracted from slides. This leads the training to be …
whole slide images