STPGANsFusion: Structure and Texture Preserving Generative Adversarial Networks for Multi-modal Medical Image Fusion
Medical images from various modalities carry diverse information. The features from these source images are combined into a single image, constituting more information content, beneficial for subsequent medical applications. Recently, deep learning (DL) based networks have demonstrated the ability to produce promising fusion results by integrating the feature extraction and preservation task with less manual interventions. However, using a single network for extracting features from multi-modal source images characterizing distinct information results in the loss of crucial diagnostic information. Addressing this problem, we present structure and texture preserving generative adversarial networks based medical image fusion method (STPGANsFusion). Initially, the textural and structural components of the source images are separated using structure gradient and texture decorrelating regularizer (SGTDR) based image decomposition for more complementary information preservation and higher robustness for the model. Next, the fusion of the structure and the texture components is carried out using two generative adversarial networks (GANs) consisting of a generator and two discriminators to get fused structure and texture components. The loss function for each GAN is framed as per the characteristic of the component being fused to minimize the loss of complementary information. The fused image is reconstructed and undergoes adaptive mask-based structure enhancement to further boost its contrast and visualization. Substantial experimentation is carried out on a wide variety of neurological images. Visual and qualitative results exhibit notable improvement in the fusion performance of the proposed method in comparison to the state-of-the-art fusion methods.
Code (0)
등록된 구현이 없습니다.
Tasks
DiagnosticSimilar Papers 제목 키워드 기반
Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction
A lot of work has been done towards reconstructing the 3D facial structure from single images by capitalizing on the power of Deep Convolutional Neural Networks (DCNNs). In the recent works, the texture features either c…
3D Face ReconstructionFace ReconstructionGenerative Adversarial NetworkVocal Bursts Intensity PredictionGANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction
In the past few years, a lot of work has been done towards reconstructing the 3D facial structure from single images by capitalizing on the power of Deep Convolutional Neural Networks (DCNNs). In the most recent works, d…
3D Face ReconstructionFace ReconstructionGenerative Adversarial NetworkVocal Bursts Intensity PredictionUnpaired Quad-Path Cycle Consistent Adversarial Networks for Single Image Defogging
Adversarial learning-based image defogging methods have been extensively studied in computer vision due to their remarkable performance. However, most existing methods have limited defogging capabilities for real cases b…
Generative Adversarial NetworkAdversarial Texture for Fooling Person Detectors in the Physical World
Nowadays, cameras equipped with AI systems can capture and analyze images to detect people automatically. However, the AI system can make mistakes when receiving deliberately designed patterns in the real world, i.e., ph…
Structure-Preserving Progressive Low-rank Image Completion for Defending Adversarial Attacks
Deep neural networks recognize objects by analyzing local image details and summarizing their information along the inference layers to derive the final decision. Because of this, they are prone to adversarial attacks. S…
Adversarial RobustnessLow-Rank Matrix CompletionMatrix Completion