Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement
We present a method to improve the visual realism of low-quality, synthetic images, e.g. OpenGL renderings. Training an unpaired synthetic-to-real translation network in image space is severely under-constrained and produces visible artifacts. Instead, we propose a semi-supervised approach that operates on the disentangled shading and albedo layers of the image. Our two-stage pipeline first learns to predict accurate shading in a supervised fashion using physically-based renderings as targets, and further increases the realism of the textures and shading with an improved CycleGAN network. Extensive evaluations on the SUNCG indoor scene dataset demonstrate that our approach yields more realistic images compared to other state-of-the-art approaches. Furthermore, networks trained on our generated "real" images predict more accurate depth and normals than domain adaptation approaches, suggesting that improving the visual realism of the images can be more effective than imposing task-specific losses.
Code (0)
등록된 구현이 없습니다.
Tasks
DisentanglementDomain AdaptationSynthetic-to-Real TranslationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation
Existing deep learning-based image deraining methods have achieved promising performance for synthetic rainy images, typically rely on the pairs of sharp images and simulated rainy counterparts. However, these methods su…
DisentanglementRain RemovalTranslationIdentity From Here, Pose From There: Self-Supervised Disentanglement and Generation of Objects Using Unlabeled Videos
We propose a novel approach that disentangles the identity and pose of objects for image generation. Our model takes as input an ID image and a pose image, and generates an output image with the identity of the ID image …
DisentanglementDiversityImage GenerationContent Disentanglement for Semantically Consistent Synthetic-to-Real Domain Adaptation
Synthetic data generation is an appealing approach to generate novel traffic scenarios in autonomous driving. However, deep learning perception algorithms trained solely on synthetic data encounter serious performance dr…
Autonomous DrivingDisentanglementDomain AdaptationSemantic Segmentation+1Unsupervised Disentanglement Learning by intervention
Recently there has been an increased interest in unsupervised learning of disentangled representations on the data generated from variation factors. Existing works rely on the assumption that the generative factors are i…
DisentanglementTranslationSemi-Supervised StyleGAN for Disentanglement Learning
Disentanglement learning is crucial for obtaining disentangled representations and controllable generation. Current disentanglement methods face several inherent limitations: difficulty with high-resolution images, prima…
DisentanglementRepresentation Learning