Interpretable Deep Multimodal Image Super-Resolution
Multimodal image super-resolution (SR) is the reconstruction of a high resolution image given a low-resolution observation with the aid of another image modality. While existing deep multimodal models do not incorporate domain knowledge about image SR, we present a multimodal deep network design that integrates coupled sparse priors and allows the effective fusion of information from another modality into the reconstruction process. Our method is inspired by a novel iterative algorithm for coupled convolutional sparse coding, resulting in an interpretable network by design. We apply our model to the super-resolution of near-infrared image guided by RGB images. Experimental results show that our model outperforms state-of-the-art methods.
Code (0)
등록된 구현이 없습니다.
Tasks
Image Super-ResolutionSuper-ResolutionSimilar Papers 제목 키워드 기반
Multimodal Deep Unfolding for Guided Image Super-Resolution
The reconstruction of a high resolution image given a low resolution observation is an ill-posed inverse problem in imaging. Deep learning methods rely on training data to learn an end-to-end mapping from a low-resolutio…
Image Super-ResolutionMultimodal Deep LearningSuper-ResolutionReconstructing Interpretable Features in Computational Super-Resolution microscopy via Regularized Latent Search
Supervised deep learning approaches can artificially increase the resolution of microscopy images by learning a mapping between two image resolutions or modalities. However, such methods often require a large set of hard…
Super-ResolutionQUSR: Quality-Aware and Uncertainty-Guided Image Super-Resolution Diffusion Model
Diffusion-based image super-resolution (ISR) has shown strong potential, but it still struggles in real-world scenarios where degradations are unknown and spatially non-uniform, often resulting in lost details or visual …
Image Super-ResolutionMultimodal Model for Computational Pathology:Representation Learning and Image Compression
Whole slide imaging (WSI) has transformed digital pathology by enabling computational analysis of gigapixel histopathology images. Recent foundation model advances have accelerated progress in computational pathology, fa…
Representation LearningImage CompressionFew-Shot LearningMultimodal Image Super-resolution via Deep Unfolding with Side Information
Deep learning methods have been successfully applied to various computer vision tasks. However, existing neural network architectures do not per se incorporate domain knowledge about the addressed problem, thus, understa…
Image Super-ResolutionSuper-Resolution