Benchmarking foundation models as feature extractors for weakly-supervised computational pathology
Advancements in artificial intelligence have driven the development of numerous pathology foundation models capable of extracting clinically relevant information. However, there is currently limited literature independently evaluating these foundation models on truly external cohorts and clinically-relevant tasks to uncover adjustments for future improvements. In this study, we benchmarked 19 histopathology foundation models on 13 patient cohorts with 6,818 patients and 9,528 slides from lung, colorectal, gastric, and breast cancers. The models were evaluated on weakly-supervised tasks related to biomarkers, morphological properties, and prognostic outcomes. We show that a vision-language foundation model, CONCH, yielded the highest performance when compared to vision-only foundation models, with Virchow2 as close second. The experiments reveal that foundation models trained on distinct cohorts learn complementary features to predict the same label, and can be fused to outperform the current state of the art. An ensemble combining CONCH and Virchow2 predictions outperformed individual models in 55% of tasks, leveraging their complementary strengths in classification scenarios. Moreover, our findings suggest that data diversity outweighs data volume for foundation models. Our work highlights actionable adjustments to improve pathology foundation models.
Code (0)
등록된 구현이 없습니다.
Tasks
BenchmarkingDiversitySimilar Papers 제목 키워드 기반
Benchmarking Pathology Feature Extractors for Whole Slide Image Classification
Weakly supervised whole slide image classification is a key task in computational pathology, which involves predicting a slide-level label from a set of image patches constituting the slide. Constructing models to solve …
Benchmarkingimage-classificationImage ClassificationSelf-Supervised Learning+1Leveraging Foundation Models for Content-Based Medical Image Retrieval in Radiology
Content-based image retrieval (CBIR) has the potential to significantly improve diagnostic aid and medical research in radiology. Current CBIR systems face limitations due to their specialization to certain pathologies, …
BenchmarkingContent-Based Image RetrievalDiagnosticImage Retrieval+2Low-resource finetuning of foundation models beats state-of-the-art in histopathology
To handle the large scale of whole slide images in computational pathology, most approaches first tessellate the images into smaller patches, extract features from these patches, and finally aggregate the feature vectors…
GPUSelf-Supervised LearningWeakly-supervised Learningwhole slide imagesWeakly Supervised Tracklet Person Re-Identification by Deep Feature-wise Mutual Learning
The scalability problem caused by the difficulty in annotating Person Re-identification(Re-ID) datasets has become a crucial bottleneck in the development of Re-ID.To address this problem, many unsupervised Re-ID methods…
Person Re-IdentificationOn Evaluating Weakly Supervised Action Segmentation Methods
Action segmentation is the task of temporally segmenting every frame of an untrimmed video. Weakly supervised approaches to action segmentation, especially from transcripts have been of considerable interest to the compu…
Action SegmentationSegmentation