REED-VAE: RE-Encode Decode Training for Iterative Image Editing with Diffusion Models
While latent diffusion models achieve impressive image editing results, their application to iterative editing of the same image is severely restricted. When trying to apply consecutive edit operations using current models, they accumulate artifacts and noise due to repeated transitions between pixel and latent spaces. Some methods have attempted to address this limitation by performing the entire edit chain within the latent space, sacrificing flexibility by supporting only a limited, predetermined set of diffusion editing operations. We present a RE-encode decode (REED) training scheme for variational autoencoders (VAEs), which promotes image quality preservation even after many iterations. Our work enables multi-method iterative image editing: users can perform a variety of iterative edit operations, with each operation building on the output of the previous one using both diffusion-based operations and conventional editing techniques. We demonstrate the advantage of REED-VAE across a range of image editing scenarios, including text-based and mask-based editing frameworks. In addition, we show how REED-VAE enhances the overall editability of images, increasing the likelihood of successful and precise edit operations. We hope that this work will serve as a benchmark for the newly introduced task of multi-method image editing. Our code and models will be available at https://github.com/galmog/REED-VAE
Code (1)
Methods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Iterative Deep Convolutional Encoder-Decoder Network for Medical Image Segmentation
In this paper, we propose a novel medical image segmentation using iterative deep learning framework. We have combined an iterative learning approach and an encoder-decoder network to improve segmentation results, which …
DecoderDeep LearningImage SegmentationMedical Image Segmentation+2MetaFun: Meta-Learning with Iterative Functional Updates
We develop a functional encoder-decoder approach to supervised meta-learning, where labeled data is encoded into an infinite-dimensional functional representation rather than a finite-dimensional one. Furthermore, rather…
DecoderFew-Shot Image ClassificationMeta-LearningGenerating Text with Deep Reinforcement Learning
We introduce a novel schema for sequence to sequence learning with a Deep Q-Network (DQN), which decodes the output sequence iteratively. The aim here is to enable the decoder to first tackle easier portions of the seque…
DecoderDeep Reinforcement Learningreinforcement-learningReinforcement Learning+2Greedy Ordering of Layer Weight Matrices in Transformers Improves Translation
Prior work has attempted to understand the internal structures and functionalities of Transformer-based encoder-decoder architectures on the level of multi-head attention and feed-forward sublayers. Interpretations have …
DecoderTranslationLarge Neighborhood Search meets Iterative Neural Constraint Heuristics
Neural networks are being increasingly used as heuristics for constraint satisfaction. These neural methods are often recurrent, learning to iteratively refine candidate assignments. In this work, we make explicit the co…