MixNMatch: Multifactor Disentanglement and Encoding for Conditional Image Generation
We present MixNMatch, a conditional generative model that learns to disentangle and encode background, object pose, shape, and texture from real images with minimal supervision, for mix-and-match image generation. We build upon FineGAN, an unconditional generative model, to learn the desired disentanglement and image generator, and leverage adversarial joint image-code distribution matching to learn the latent factor encoders. MixNMatch requires bounding boxes during training to model background, but requires no other supervision. Through extensive experiments, we demonstrate MixNMatch's ability to accurately disentangle, encode, and combine multiple factors for mix-and-match image generation, including sketch2color, cartoon2img, and img2gif applications. Our code/models/demo can be found at https://github.com/Yuheng-Li/MixNMatch
Code (3)
Tasks
Conditional Image GenerationDisentanglementImage GenerationSimilar Papers 제목 키워드 기반
Multifactor Sequential Disentanglement via Structured Koopman Autoencoders
Disentangling complex data to its latent factors of variation is a fundamental task in representation learning. Existing work on sequential disentanglement mostly provides two factor representations, i.e., it separates t…
DisentanglementInductive BiasRepresentation LearningGenerative Adversarial Networks and Adversarial Autoencoders: Tutorial and Survey
This is a tutorial and survey paper on Generative Adversarial Network (GAN), adversarial autoencoders, and their variants. We start with explaining adversarial learning and the vanilla GAN. Then, we explain the condition…
Dimensionality ReductionFace GenerationGenerative Adversarial NetworkImage Generation+4Representation Disentanglement for Multi-modal brain MR Analysis
Multi-modal MRIs are widely used in neuroimaging applications since different MR sequences provide complementary information about brain structures. Recent works have suggested that multi-modal deep learning analysis can…
Brain Tumor SegmentationDisentanglementTumor SegmentationVariational Learning with Disentanglement-PyTorch
Unsupervised learning of disentangled representations is an open problem in machine learning. The Disentanglement-PyTorch library is developed to facilitate research, implementation, and testing of new variational algori…
DisentanglementSchedulingVariational Learning with Disentanglement-PyTorch
Unsupervised learning of disentangled representations is an open problem in machine learning. The Disentanglement-PyTorch library is developed to facilitate research, implementation, and testing of new variational algori…
DisentanglementScheduling