Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Network
Magnetic resonance imaging (MRI) is being increasingly utilized to assess, diagnose, and plan treatment for a variety of diseases. The ability to visualize tissue in varied contrasts in the form of MR pulse sequences in a single scan provides valuable insights to physicians, as well as enabling automated systems performing downstream analysis. However many issues like prohibitive scan time, image corruption, different acquisition protocols, or allergies to certain contrast materials may hinder the process of acquiring multiple sequences for a patient. This poses challenges to both physicians and automated systems since complementary information provided by the missing sequences is lost. In this paper, we propose a variant of generative adversarial network (GAN) capable of leveraging redundant information contained within multiple available sequences in order to generate one or more missing sequences for a patient scan. The proposed network is designed as a multi-input, multi-output network which combines information from all the available pulse sequences, implicitly infers which sequences are missing, and synthesizes the missing ones in a single forward pass. We demonstrate and validate our method on two brain MRI datasets each with four sequences, and show the applicability of the proposed method in simultaneously synthesizing all missing sequences in any possible scenario where either one, two, or three of the four sequences may be missing. We compare our approach with competing unimodal and multi-modal methods, and show that we outperform both quantitatively and qualitatively.
Code (0)
등록된 구현이 없습니다.
Tasks
Generative Adversarial NetworkSimilar Papers 제목 키워드 기반
PEPSI: Pathology-Enhanced Pulse-Sequence-Invariant Representations for Brain MRI
Remarkable progress has been made by data-driven machine-learning methods in the analysis of MRI scans. However, most existing MRI analysis approaches are crafted for specific MR pulse sequences (MR contrasts) and usuall…
AnatomyImage GenerationRepresentation LearningMRI Pulse Sequence Integration for Deep-Learning Based Brain Metastasis Segmentation
Magnetic resonance (MR) imaging is an essential diagnostic tool in clinical medicine. Recently, a variety of deep learning methods have been applied to segmentation tasks in medical images, with promising results for com…
DiagnosticSmall Data Image ClassificationGenerative Adversarial Networks for Brain Images Synthesis: A Review
In medical imaging, image synthesis is the estimation process of one image (sequence, modality) from another image (sequence, modality). Since images with different modalities provide diverse biomarkers and capture vario…
Deep LearningGenerative Adversarial NetworkImage GenerationRadiological image synthesis using cycle-consistent generative adversarial network
In medical imaging, image synthesis is the estimation process of one image (sequence, modality) from another image (sequence, modality). Since images with different modalities provide diverse biomarkers and capture vario…
Deep LearningGenerative Adversarial NetworkImage GenerationDiamondGAN: Unified Multi-Modal Generative Adversarial Networks for MRI Sequences Synthesis
Synthesizing MR imaging sequences is highly relevant in clinical practice, as single sequences are often missing or are of poor quality (e.g. due to motion). Naturally, the idea arises that a target modality would benefi…
Image Generation