Modality-Projection Universal Model for Comprehensive Full-Body Medical Imaging Segmentation
The integration of deep learning in medical imaging has shown great promise for enhancing diagnostic, therapeutic, and research outcomes. However, applying universal models across multiple modalities remains challenging due to the inherent variability in data characteristics. This study aims to introduce and evaluate a Modality Projection Universal Model (MPUM). MPUM employs a novel modality-projection strategy, which allows the model to dynamically adjust its parameters to optimize performance across different imaging modalities. The MPUM demonstrated superior accuracy in identifying anatomical structures, enabling precise quantification for improved clinical decision-making. It also identifies metabolic associations within the brain-body axis, advancing research on brain-body physiological correlations. Furthermore, MPUM's unique controller-based convolution layer enables visualization of saliency maps across all network layers, significantly enhancing the model's interpretability.
Code (1)
Tasks
Decision MakingDiagnosticMedical Image SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Concept-Centric Approach to Multi-Modality Learning
In an effort to create a more efficient AI system, we introduce a new multi-modality learning framework that leverages a modality-agnostic concept space possessing abstract knowledge and a set of modality-specific projec…
Image-text matchingQuestion AnsweringText MatchingVisual Question AnsweringOneLLM: One Framework to Align All Modalities with Language
Multimodal large language models (MLLMs) have gained significant attention due to their strong multimodal understanding capability. However, existing works rely heavily on modality-specific encoders, which usually differ…
AllQuestion AnsweringVisual Question AnsweringDeveloping Foundation Models for Universal Segmentation from 3D Whole-Body Positron Emission Tomography
Positron emission tomography (PET) is a key nuclear medicine imaging modality that visualizes radiotracer distributions to quantify in vivo physiological and metabolic processes, playing an irreplaceable role in disease …
Lesion SegmentationPrompt EngineeringUniversal Scene Graph Generation
Scene graph (SG) representations can neatly and efficiently describe scene semantics, which has driven sustained intensive research in SG generation. In the real world, multiple modalities often coexist, with different t…
Graph GenerationScene Graph GenerationSegBook: A Simple Baseline and Cookbook for Volumetric Medical Image Segmentation
Computed Tomography (CT) is one of the most popular modalities for medical imaging. By far, CT images have contributed to the largest publicly available datasets for volumetric medical segmentation tasks, covering full-b…
Computed Tomography (CT)Image SegmentationLesion DetectionMedical Image Segmentation+3