Panoptic-aware Image-to-Image Translation
Despite remarkable progress in image translation, the complex scene with multiple discrepant objects remains a challenging problem. The translated images have low fidelity and tiny objects in fewer details causing unsatisfactory performance in object recognition. Without thorough object perception (i.e., bounding boxes, categories, and masks) of images as prior knowledge, the style transformation of each object will be difficult to track in translation. We propose panoptic-aware generative adversarial networks (PanopticGAN) for image-to-image translation together with a compact panoptic segmentation dataset. The panoptic perception (i.e., foreground instances and background semantics of the image scene) is extracted to achieve alignment between object content codes of the input domain and panoptic-level style codes sampled from the target style space, then refined by a proposed feature masking module for sharping object boundaries. The image-level combination between content and sampled style codes is also merged for higher fidelity image generation. Our proposed method was systematically compared with different competing methods and obtained significant improvement in both image quality and object recognition performance.
Code (0)
등록된 구현이 없습니다.
Tasks
Image GenerationImage-to-Image TranslationObjectObject RecognitionPanoptic SegmentationTranslationSimilar Papers 제목 키워드 기반
Region and Object based Panoptic Image Synthesis through Conditional GANs
Image-to-image translation is significant to many computer vision and machine learning tasks such as image synthesis and video synthesis. It has primary applications in the graphics editing and animation industries. With…
Image GenerationImage-to-Image TranslationTranslationImproving Panoptic Segmentation for Nighttime or Low-Illumination Urban Driving Scenes
Autonomous vehicles and driving systems use scene parsing as an essential tool to understand the surrounding environment. Panoptic segmentation is a state-of-the-art technique which proves to be pivotal in this use case.…
Autonomous VehiclesPanoptic SegmentationScene ParsingSegmentationPanoptic-based Image Synthesis
Conditional image synthesis for generating photorealistic images serves various applications for content editing to content generation. Previous conditional image synthesis algorithms mostly rely on semantic maps, and of…
Image GenerationPanoptic Captioning: Seeking An Equivalency Bridge for Image and Text
This work introduces panoptic captioning, a novel task striving to seek the minimum text equivalence of images. We take the first step towards panoptic captioning by formulating it as a task of generating a comprehensive…
Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representation
We present Panoptic Neural Fields (PNF), an object-aware neural scene representation that decomposes a scene into a set of objects (things) and background (stuff). Each object is represented by an oriented 3D bounding bo…
2D Panoptic Segmentation3D scene EditingDepth EstimationDepth Prediction+3