Harmonizing Maximum Likelihood with GANs for Multimodal Conditional Generation
Recent advances in conditional image generation tasks, such as image-to-image translation and image inpainting, are largely accounted to the success of conditional GAN models, which are often optimized by the joint use of the GAN loss with the reconstruction loss. However, we reveal that this training recipe shared by almost all existing methods causes one critical side effect: lack of diversity in output samples. In order to accomplish both training stability and multimodal output generation, we propose novel training schemes with a new set of losses named moment reconstruction losses that simply replace the reconstruction loss. We show that our approach is applicable to any conditional generation tasks by performing thorough experiments on image-to-image translation, super-resolution and image inpainting using Cityscapes and CelebA dataset. Quantitative evaluations also confirm that our methods achieve a great diversity in outputs while retaining or even improving the visual fidelity of generated samples.
Code (0)
등록된 구현이 없습니다.
Tasks
Conditional Image GenerationDiversityImage GenerationImage InpaintingImage-to-Image TranslationSuper-ResolutionTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Multimodal Image Synthesis with Conditional Implicit Maximum Likelihood Estimation
Many tasks in computer vision and graphics fall within the framework of conditional image synthesis. In recent years, generative adversarial nets (GANs) have delivered impressive advances in quality of synthesized images…
Image GenerationImage Super-ResolutionSuper-ResolutionCascading Modular Network (CAM-Net) for Multimodal Image Synthesis
Deep generative models such as GANs have driven impressive advances in conditional image synthesis in recent years. A persistent challenge has been to generate diverse versions of output images from the same input image,…
Image GenerationFlow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models
Adversarial learning of probabilistic models has recently emerged as a promising alternative to maximum likelihood. Implicit models such as generative adversarial networks (GAN) often generate better samples compared to …
Generative Adversarial NetworkModality for Scenario Analysis and Maximum Likelihood Allocation
We study the variability of a risk from the statistical viewpoint of multimodality of the conditional loss distribution given that the aggregate loss equals an exogenously provided capital. This conditional distribution …
Maximum Likelihood Estimation for Multimodal Learning with Missing Modality
Multimodal learning has achieved great successes in many scenarios. Compared with unimodal learning, it can effectively combine the information from different modalities to improve the performance of learning tasks. In r…