A Learnable Variational Model for Joint Multimodal MRI Reconstruction and Synthesis
Generating multi-contrasts/modal MRI of the same anatomy enriches diagnostic information but is limited in practice due to excessive data acquisition time. In this paper, we propose a novel deep-learning model for joint reconstruction and synthesis of multi-modal MRI using incomplete k-space data of several source modalities as inputs. The output of our model includes reconstructed images of the source modalities and high-quality image synthesized in the target modality. Our proposed model is formulated as a variational problem that leverages several learnable modality-specific feature extractors and a multimodal synthesis module. We propose a learnable optimization algorithm to solve this model, which induces a multi-phase network whose parameters can be trained using multi-modal MRI data. Moreover, a bilevel-optimization framework is employed for robust parameter training. We demonstrate the effectiveness of our approach using extensive numerical experiments.
Code (0)
등록된 구현이 없습니다.
Tasks
AnatomyBilevel OptimizationDiagnosticMRI ReconstructionSimilar Papers 제목 키워드 기반
Optimization-Based Deep learning methods for Magnetic Resonance Imaging Reconstruction and Synthesis
This dissertation is devoted to provide advanced nonconvex nonsmooth variational models of (Magnetic Resonance Image) MRI reconstruction, efficient learnable image reconstruction algorithms and parameter training algorit…
compressed sensingImage ReconstructionMeta-LearningMRI ReconstructionMultimodal synthesis of MRI and tabular data with diffusion in a joint latent space via cross-attention
We propose a multimodal latent diffusion model that jointly synthesizes volumetric magnetic resonance imaging (MRI) and tabular clinical data within a shared latent space via cross-attention. This approach enables cohere…
Representation LearningImage GenerationShaLa: Multimodal Shared Latent Space Modelling
This paper presents a novel generative framework for learning shared latent representations across multimodal data. Many advanced multimodal methods focus on capturing all combinations of modality-specific details across…
CoVAE: correlated multimodal generative modeling
Multimodal Variational Autoencoders have emerged as a popular tool to extract effective representations from rich multimodal data. However, such models rely on fusion strategies in latent space that destroy the joint sta…
On learning adaptive acquisition policies for undersampled multi-coil MRI reconstruction
Most current approaches to undersampled multi-coil MRI reconstruction focus on learning the reconstruction model for a fixed, equidistant acquisition trajectory. In this paper, we study the problem of joint learning of t…
MRI ReconstructionSSIM