Toward Unpaired Multi-modal Medical Image Segmentation via Learning Structured Semantic Consistency
Integrating multi-modal data to promote medical image analysis has recently gained great attention. This paper presents a novel scheme to learn the mutual benefits of different modalities to achieve better segmentation results for unpaired multi-modal medical images. Our approach tackles two critical issues of this task from a practical perspective: (1) how to effectively learn the semantic consistencies of various modalities (e.g., CT and MRI), and (2) how to leverage the above consistencies to regularize the network learning while preserving its simplicity. To address (1), we leverage a carefully designed External Attention Module (EAM) to align semantic class representations and their correlations of different modalities. To solve (2), the proposed EAM is designed as an external plug-and-play one, which can be discarded once the model is optimized. We have demonstrated the effectiveness of the proposed method on two medical image segmentation scenarios: (1) cardiac structure segmentation, and (2) abdominal multi-organ segmentation. Extensive results show that the proposed method outperforms its counterparts by a wide margin.
Code (0)
등록된 구현이 없습니다.
Tasks
Image SegmentationMedical Image AnalysisMedical Image SegmentationOrgan SegmentationSegmentationSemantic SegmentationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Partially Supervised Unpaired Multi-Modal Learning for Label-Efficient Medical Image Segmentation
Unpaired Multi-Modal Learning (UMML) which leverages unpaired multi-modal data to boost model performance on each individual modality has attracted a lot of research interests in medical image analysis. However, existing…
Image SegmentationMedical Image AnalysisMedical Image SegmentationOrgan Segmentation+3Unified Attentional Generative Adversarial Network for Brain Tumor Segmentation From Multimodal Unpaired Images
In medical applications, the same anatomical structures may be observed in multiple modalities despite the different image characteristics. Currently, most deep models for multimodal segmentation rely on paired registere…
Brain Tumor SegmentationGenerative Adversarial NetworkSegmentationTranslation+1MulModSeg: Enhancing Unpaired Multi-Modal Medical Image Segmentation with Modality-Conditioned Text Embedding and Alternating Training
In the diverse field of medical imaging, automatic segmentation has numerous applications and must handle a wide variety of input domains, such as different types of Computed Tomography (CT) scans and Magnetic Resonance …
Computed Tomography (CT)Image SegmentationMedical Image SegmentationSegmentation+2Shape-consistent Generative Adversarial Networks for multi-modal Medical segmentation maps
Image translation across domains for unpaired datasets has gained interest and great improvement lately. In medical imaging, there are multiple imaging modalities, with very different characteristics. Our goal is to use …
Generative Adversarial NetworkSegmentationSemantic SegmentationTranslationDeep Class-Specific Affinity-Guided Convolutional Network for Multimodal Unpaired Image Segmentation
Multi-modal medical image segmentation plays an essential role in clinical diagnosis. It remains challenging as the input modalities are often not well-aligned spatially. Existing learning-based methods mainly consider s…
Image SegmentationMedical Image SegmentationSegmentationSemantic Segmentation