Zero-shot Text-driven Physically Interpretable Face Editing

2023-08-11 · Yapeng Meng, Songru Yang, Xu Hu, Rui Zhao, Lincheng Li, Zhenwei Shi, Zhengxia Zou

This paper proposes a novel and physically interpretable method for face editing based on arbitrary text prompts. Different from previous GAN-inversion-based face editing methods that manipulate the latent space of GANs, or diffusion-based methods that model image manipulation as a reverse diffusion process, we regard the face editing process as imposing vector flow fields on face images, representing the offset of spatial coordinates and color for each image pixel. Under the above-proposed paradigm, we represent the vector flow field in two ways: 1) explicitly represent the flow vectors with rasterized tensors, and 2) implicitly parameterize the flow vectors as continuous, smooth, and resolution-agnostic neural fields, by leveraging the recent advances of implicit neural representations. The flow vectors are iteratively optimized under the guidance of the pre-trained Contrastive Language-Image Pretraining~(CLIP) model by maximizing the correlation between the edited image and the text prompt. We also propose a learning-based one-shot face editing framework, which is fast and adaptable to any text prompt input. Our method can also be flexibly extended to real-time video face editing. Compared with state-of-the-art text-driven face editing methods, our method can generate physically interpretable face editing results with high identity consistency and image quality. Our code will be made publicly available.

📄 PDF Abstract BibTeX arXiv:2308.05976

Code (0)

등록된 구현이 없습니다.

Tasks

Image Manipulation

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Interpretable Zero-shot Referring Expression Comprehension with Query-driven Scene Graphs

2026-03-26 · Yike Wu, Necva Bolucu, Stephen Wan, Dadong Wang 외 arxiv

Zero-shot referring expression comprehension (REC) aims to locate target objects in images given natural language queries without relying on task-specific training data, demanding strong visual understanding capabilities…

Natural Language Queries Referring Expression Scene Understanding

Differentiable Inverse Graphics for Zero-shot Scene Reconstruction and Robot Grasping

2026-02-04 · Octavio Arriaga, Proneet Sharma, Jichen Guo, Marc Otto 외 arxiv

Operating effectively in novel real-world environments requires robotic systems to estimate and interact with previously unseen objects. Current state-of-the-art models address this challenge by using large amounts of tr…

Pose Estimation

LLM-FS: Zero-Shot Feature Selection for Effective and Interpretable Malware Detection

2026-02-10 · Naveen Gill, Ajvad Haneef K, Madhu Kumar S D arxiv

Feature selection (FS) remains essential for building accurate and interpretable detection models, particularly in high-dimensional malware datasets. Conventional FS methods such as Extra Trees, Variance Threshold, Tree-…

Malware Detection

TopoBrick: Agentic Topology Sampling of Exogenous Variables for Zero-Shot Building IoT Forecasting

2026-07-07 · Xiachong Lin, Du Yin, Arian Prabowo, Hao Xue 외 arxiv

Building sensors are embedded in physical topology, spatial hierarchy, and operational context, yet existing forecasters often treat them as isolated time series or rely on fixed covariate sets. We present TopoBrick, a t…

Knowledge Graphs

"Oh LLM, I'm Asking Thee, Please Give Me a Decision Tree": Zero-Shot Decision Tree Induction and Embedding with Large Language Models

2024-09-27 · Ricardo Knauer, Mario Koddenbrock, Raphael Wallsberger, Nicholas M. Brisson 외

Large language models (LLMs) provide powerful means to leverage prior knowledge for predictive modeling when data is limited. In this work, we demonstrate how LLMs can use their compressed world knowledge to generate int…

Interpretable Machine Learning World Knowledge