SkrGAN: Sketching-rendering Unconditional Generative Adversarial Networks for Medical Image Synthesis
Generative Adversarial Networks (GANs) have the capability of synthesizing images, which have been successfully applied to medical image synthesis tasks. However, most of existing methods merely consider the global contextual information and ignore the fine foreground structures, e.g., vessel, skeleton, which may contain diagnostic indicators for medical image analysis. Inspired by human painting procedure, which is composed of stroking and color rendering steps, we propose a Sketching-rendering Unconditional Generative Adversarial Network (SkrGAN) to introduce a sketch prior constraint to guide the medical image generation. In our SkrGAN, a sketch guidance module is utilized to generate a high quality structural sketch from random noise, then a color render mapping is used to embed the sketch-based representations and resemble the background appearances. Experimental results show that the proposed SkrGAN achieves the state-of-the-art results in synthesizing images for various image modalities, including retinal color fundus, X-Ray, Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). In addition, we also show that the performances of medical image segmentation method have been improved by using our synthesized images as data augmentation.
Code (0)
등록된 구현이 없습니다.
Tasks
Computed Tomography (CT)Data AugmentationDiagnosticGenerative Adversarial NetworkImage GenerationImage SegmentationMedical Image AnalysisMedical Image GenerationMedical Image SegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications
The generative adversarial network (GAN) framework has emerged as a powerful tool for various image and video synthesis tasks, allowing the synthesis of visual content in an unconditional or input-conditional manner. It …
Generative Adversarial NetworkNeural RenderingTranslation3inGAN: Learning a 3D Generative Model from Images of a Self-similar Scene
We introduce 3inGAN, an unconditional 3D generative model trained from 2D images of a single self-similar 3D scene. Such a model can be used to produce 3D "remixes" of a given scene, by mapping spatial latent codes into …
Generative Adversarial Network3D-Aware Video Generation
Generative models have emerged as an essential building block for many image synthesis and editing tasks. Recent advances in this field have also enabled high-quality 3D or video content to be generated that exhibits eit…
Image GenerationVideo GenerationTheoretical Insights into the Use of Structural Similarity Index In Generative Models and Inferential Autoencoders
Generative models and inferential autoencoders mostly make use of $\ell_2$ norm in their optimization objectives. In order to generate perceptually better images, this short paper theoretically discusses how to use Struc…
Dimensionality ReductionImage GenerationImage Quality AssessmentSSIMFrom Scratch to Sketch: Deep Decoupled Hierarchical Reinforcement Learning for Robotic Sketching Agent
We present an automated learning framework for a robotic sketching agent that is capable of learning stroke-based rendering and motor control simultaneously. We formulate the robotic sketching problem as a deep decoupled…
Hierarchical Reinforcement Learningreinforcement-learningReinforcement Learning (RL)