Controllable Semantic Image Inpainting
We develop a method for user-controllable semantic image inpainting: Given an arbitrary set of observed pixels, the unobserved pixels can be imputed in a user-controllable range of possibilities, each of which is semantically coherent and locally consistent with the observed pixels. We achieve this using a deep generative model bringing together: an encoder which can encode an arbitrary set of observed pixels, latent variables which are trained to represent disentangled factors of variations, and a bidirectional PixelCNN model. We experimentally demonstrate that our method can generate plausible inpainting results matching the user-specified semantics, but is still coherent with observed pixels. We justify our choices of architecture and training regime through more experiments.
Code (0)
등록된 구현이 없습니다.
Tasks
Image InpaintingSimilar Papers 제목 키워드 기반
Improving Text-guided Object Inpainting with Semantic Pre-inpainting
Recent years have witnessed the success of large text-to-image diffusion models and their remarkable potential to generate high-quality images. The further pursuit of enhancing the editability of images has sparked signi…
DenoisingObjectReference-Guided Diffusion Inpainting For Multimodal Counterfactual Generation
Safety-critical applications, such as autonomous driving and medical image analysis, require extensive multimodal data for rigorous testing. Synthetic data methods are gaining prominence due to the cost and complexity of…
Synthetic Data GenerationAutonomous DrivingReference-Guided Large-Scale Face Inpainting with Identity and Texture Control
Face inpainting aims at plausibly predicting missing pixels of face images within a corrupted region. Most existing methods rely on generative models learning a face image distribution from a big dataset, which produces …
Facial InpaintingMObI: Multimodal Object Inpainting Using Diffusion Models
Safety-critical applications, such as autonomous driving, require extensive multimodal data for rigorous testing. Methods based on synthetic data are gaining prominence due to the cost and complexity of gathering real-wo…
Autonomous DrivingObjectMagicEraser: Erasing Any Objects via Semantics-Aware Control
The traditional image inpainting task aims to restore corrupted regions by referencing surrounding background and foreground. However, the object erasure task, which is in increasing demand, aims to erase objects and gen…
Image InpaintingObjectTexture Synthesis