Image Generation and Translation with Disentangled Representations
Generative models have made significant progress in the tasks of modeling complex data distributions such as natural images. The introduction of Generative Adversarial Networks (GANs) and auto-encoders lead to the possibility of training on big data sets in an unsupervised manner. However, for many generative models it is not possible to specify what kind of image should be generated and it is not possible to translate existing images into new images of similar domains. Furthermore, models that can perform image-to-image translation often need distinct models for each domain, making it hard to scale these systems to multiple domain image-to-image translation. We introduce a model that can do both, controllable image generation and image-to-image translation between multiple domains. We split our image representation into two parts encoding unstructured and structured information respectively. The latter is designed in a disentangled manner, so that different parts encode different image characteristics. We train an encoder to encode images into these representations and use a small amount of labeled data to specify what kind of information should be encoded in the disentangled part. A generator is trained to generate images from these representations using the characteristics provided by the disentangled part of the representation. Through this we can control what kind of images the generator generates, translate images between different domains, and even learn unknown data-generating factors while only using one single model.
Code (0)
등록된 구현이 없습니다.
Tasks
Conditional Image GenerationFace GenerationImage GenerationImage-to-Image TranslationRepresentation LearningTranslationSimilar Papers 제목 키워드 기반
Learning Disentangled Representations for Image Translation
Recent approaches for unsupervised image translation are strongly reliant on generative adversarial training and architectural locality constraints. Despite their appealing results, it can be easily observed that the lea…
DisentanglementDiversityTranslationRotation and Translation Invariant Representation Learning with Implicit Neural Representations
In many computer vision applications, images are acquired with arbitrary or random rotations and translations, and in such setups, it is desirable to obtain semantic representations disentangled from the image orientatio…
ClusteringRepresentation LearningTranslationSelf-Supervised 2D Image to 3D Shape Translation with Disentangled Representations
We present a framework to translate between 2D image views and 3D object shapes. Recent progress in deep learning enabled us to learn structure-aware representations from a scene. However, the existing literature assumes…
Image to 3DTranslationUnsupervised Object Representation Learning using Translation and Rotation Group Equivariant VAE
In many imaging modalities, objects of interest can occur in a variety of locations and poses (i.e. are subject to translations and rotations in 2d or 3d), but the location and pose of an object does not change its seman…
Learning Semantic RepresentationsObjectObject DetectionPose Prediction+2Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation
Existing deep learning-based image deraining methods have achieved promising performance for synthetic rainy images, typically rely on the pairs of sharp images and simulated rainy counterparts. However, these methods su…
DisentanglementRain RemovalTranslation