VISTA3D: Versatile Imaging SegmenTation and Annotation model for 3D Computed Tomography
Medical image segmentation is a core component of precision medicine, and 3D computed tomography (CT) is one of the most important imaging techniques. A highly accurate and clinically applicable segmentation foundation model will greatly facilitate clinicians and researchers using CT images. Although existing foundation models have attracted great interest, none are adequate for 3D CT, either because they lack accurate automatic segmentation for large cohort analysis or the ability to segment novel classes. An ideal segmentation solution should possess two features: accurate out-of-the-box performance covering major organ classes, and effective adaptation or zero-shot ability to novel structures. To achieve this goal, we introduce Versatile Imaging SegmenTation and Annotation model (VISTA3D). VISTA3D is trained systematically on 11454 volumes and provides accurate out-of-the-box segmentation for 127 common types of human anatomical structures and various lesions. Additionally, VISTA3D supports 3D interactive segmentation, allowing convenient editing of automatic results and achieving state-of-the-art annotation results on unseen classes. The novel model design and training recipe represent a promising step toward developing a versatile medical image foundation model and will serve as a valuable foundation for CT image analysis. Code and model weights are available at https://github.com/Project-MONAI/VISTA
Code (1)
Tasks
Computed Tomography (CT)Image SegmentationInteractive SegmentationMedical Image SegmentationSegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imaging
Foundation models for interactive segmentation in 2D natural images and videos have sparked significant interest in building 3D foundation models for medical imaging. However, the domain gaps and clinical use cases f…
Interactive SegmentationSegmentationThe Mapillary Vistas Dataset for Semantic Understanding of Street Scenes
The Mapillary Vistas Dataset is a novel, large-scale street-level image dataset containing 25,000 high-resolution images annotated into 66 object categories with additional, instance-specific labels for 37 classes. Annot…
DiversityImage SegmentationInstance Segmentationroad scene understanding+3VISTA-PATH: An interactive foundation model for pathology image segmentation and quantitative analysis in computational pathology
Accurate semantic segmentation for histopathology image is crucial for quantitative tissue analysis and downstream clinical modeling. Recent segmentation foundation models have improved generalization through large-scale…
Semantic SegmentationImage SegmentationVISTA-OCR: Towards generative and interactive end to end OCR models
We introduce \textbf{VISTA-OCR} (Vision and Spatially-aware Text Analysis OCR), a lightweight architecture that unifies text detection and recognition within a single generative model. Unlike conventional methods that re…
DecoderOptical Character Recognition (OCR)Text DetectionA Short Review and Evaluation of SAM2's Performance in 3D CT Image Segmentation
Since the release of Segment Anything 2 (SAM2), the medical imaging community has been actively evaluating its performance for 3D medical image segmentation. However, different studies have employed varying evaluation pi…
Image SegmentationMedical Image SegmentationSemantic Segmentation