MDP: A Generalized Framework for Text-Guided Image Editing by Manipulating the Diffusion Path
Image generation using diffusion can be controlled in multiple ways. In this paper, we systematically analyze the equations of modern generative diffusion networks to propose a framework, called MDP, that explains the design space of suitable manipulations. We identify 5 different manipulations, including intermediate latent, conditional embedding, cross attention maps, guidance, and predicted noise. We analyze the corresponding parameters of these manipulations and the manipulation schedule. We show that some previous editing methods fit nicely into our framework. Particularly, we identified one specific configuration as a new type of control by manipulating the predicted noise, which can perform higher-quality edits than previous work for a variety of local and global edits.
Code (2)
Tasks
Image Generationtext-guided-image-editingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps
Diffusion distillation represents a highly promising direction for achieving faithful text-to-image generation in a few sampling steps. However, despite recent successes, existing distilled models still do not provide th…
Image GenerationImage Manipulationtext-guided-image-editingText to Image Generation+1SwiftEdit: Lightning Fast Text-Guided Image Editing via One-Step Diffusion
Recent advances in text-guided image editing enable users to perform image edits through simple text inputs, leveraging the extensive priors of multi-step diffusion-based text-to-image models. However, these methods ofte…
Image Reconstructiontext-guided-image-editingForgedit: Text Guided Image Editing via Learning and Forgetting
Text-guided image editing on real or synthetic images, given only the original image itself and the target text prompt as inputs, is a very general and challenging task. It requires an editing model to estimate by itself…
text-guided-image-editingGIE-Bench: Towards Grounded Evaluation for Text-Guided Image Editing
Editing images using natural language instructions has become a natural and expressive way to modify visual content; yet, evaluating the performance of such models remains challenging. Existing evaluation approaches ofte…
Instruction FollowingMultiple-choicetext-guided-image-editingtext similarityA Survey of Multimodal-Guided Image Editing with Text-to-Image Diffusion Models
Image editing aims to edit the given synthetic or real image to meet the specific requirements from users. It is widely studied in recent years as a promising and challenging field of Artificial Intelligence Generative C…
Video Editing