CoIE: Chain-of-Instruct Editing for Multi-Attribute Face Manipulation
Current text-to-image editing models often encounter challenges with smoothly manipulating multiple attributes using a single instruction. Taking inspiration from the Chain-of-Thought prompting technique utilized in language models, we present an innovative concept known as Chain-of-Instruct Editing (CoIE), which enhances the capabilities of these models through step-by-step editing using a series of instructions. In particular, in the context of face manipulation, we leverage the contextual learning abilities of a pretrained Large Language Model (LLM), such as GPT-4, to generate a sequence of instructions from the original input, utilizing a purpose-designed 1-shot template. To further improve the precision of each editing step, we conduct fine-tuning on the editing models using our self-constructed instruction-guided face editing dataset, Instruct-CelebA. And additionally, we incorporate a super-resolution module to mitigate the adverse effects of editability and quality degradation. Experimental results across various challenging cases confirm the significant boost in multi-attribute facial image manipulation using chain-of-instruct editing. This is evident in enhanced editing success rates, measured by CLIPSim and Coverage metrics, improved by 17.86% and 85.45% respectively, and heightened controllability indicated by Preserve L1 and Quality metrics, improved by 11.58% and 4.93% respectively.
Code (0)
등록된 구현이 없습니다.
Tasks
AttributeImage ManipulationLanguage ModellingLarge Language ModelSuper-ResolutionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Instruction-based Image Editing with Planning, Reasoning, and Generation
Editing images via instruction provides a natural way to generate interactive content, but it is a big challenge due to the higher requirement of scene understanding and generation. Prior work utilizes a chain of large l…
Object SegmentationScene UnderstandingImage EditingEmoKGEdit: Training-free Affective Injection via Visual Cue Transformation
Existing image emotion editing methods struggle to disentangle emotional cues from latent content representations, often yielding weak emotional expression and distorted visual structures. To bridge this gap, we propose …
NumeriKontrol: Adding Numeric Control to Diffusion Transformers for Instruction-based Image Editing
Instruction-based image editing enables intuitive manipulation through natural language commands. However, text instructions alone often lack the precision required for fine-grained control over edit intensity. We introd…
Image ManipulationImage EditingComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependencies
Text-driven image editing has achieved remarkable success in following single instructions. However, real-world scenarios often involve complex, multi-step instructions, particularly ``chain'' instructions where operatio…
BenchmarkingSliderEdit: Continuous Image Editing with Fine-Grained Instruction Control
Instruction-based image editing models have recently achieved impressive performance, enabling complex edits to an input image from a multi-instruction prompt. However, these models apply each instruction in the prompt w…
Text-to-Image GenerationImage ManipulationImage Editing