SinIR: Efficient General Image Manipulation with Single Image Reconstruction
We propose SinIR, an efficient reconstruction-based framework trained on a single natural image for general image manipulation, including super-resolution, editing, harmonization, paint-to-image, photo-realistic style transfer, and artistic style transfer. We train our model on a single image with cascaded multi-scale learning, where each network at each scale is responsible for image reconstruction. This reconstruction objective greatly reduces the complexity and running time of training, compared to the GAN objective. However, the reconstruction objective also exacerbates the output quality. Therefore, to solve this problem, we further utilize simple random pixel shuffling, which also gives control over manipulation, inspired by the Denoising Autoencoder. With quantitative evaluation, we show that SinIR has competitive performance on various image manipulation tasks. Moreover, with a much simpler training objective (i.e., reconstruction), SinIR is trained 33.5 times faster than SinGAN (for 500 X 500 images) that solves similar tasks. Our code is publicly available at github.com/YooJiHyeong/SinIR.
Code (1)
Tasks
DenoisingImage ManipulationImage ReconstructionStyle TransferSuper-ResolutionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
TAFIM: Targeted Adversarial Attacks against Facial Image Manipulations
Face manipulation methods can be misused to affect an individual's privacy or to spread disinformation. To this end, we introduce a novel data-driven approach that produces image-specific perturbations which are embedded…
DeepFake DetectionDetecting Image ManipulationImage ManipulationImage Shape Manipulation from a Single Augmented Training Sample
In this paper, we present DeepSIM, a generative model for conditional image manipulation based on a single image. We find that extensive augmentation is key for enabling single image training, and incorporate the use of …
Image GenerationImage ManipulationImage-to-Image TranslationSketch-to-Image TranslationImage Shape Manipulation from a Single Augmented Training Sample
In this paper, we present DeepSIM, a generative model for conditional image manipulation based on a single image. We find that extensive augmentation is key for enabling single image training, and incorporate the use of …
Image GenerationImage ManipulationImage-to-Image TranslationSketch-to-Image TranslationLDEdit: Towards Generalized Text Guided Image Manipulation via Latent Diffusion Models
Research in vision-language models has seen rapid developments off-late, enabling natural language-based interfaces for image generation and manipulation. Many existing text guided manipulation techniques are restricted …
Image GenerationImage ManipulationStyle TransferText to Image Generation+1Detecting Deepfakes with Self-Blended Images
In this paper, we present novel synthetic training data called self-blended images (SBIs) to detect deepfakes. SBIs are generated by blending pseudo source and target images from single pristine images, reproducing commo…
DeepFake Detection