paper-with-me

Papers

Shaping a Stabilized Video by Mitigating Unintended Changes for Concept-Augmented Video Editing

2024-10-16 · Mingce Guo, Jingxuan He, Shengeng Tang, Zhangye Wang, Lechao Cheng

Text-driven video editing utilizing generative diffusion models has garnered significant attention due to their potential applications. However, existing approaches are constrained by the limited word embeddings provided in pre-training, which hinders nuanced editing targeting open concepts with specific attributes. Directly altering the keywords in target prompts often results in unintended disruptions to the attention mechanisms. To achieve more flexible editing easily, this work proposes an improved concept-augmented video editing approach that generates diverse and stable target videos flexibly by devising abstract conceptual pairs. Specifically, the framework involves concept-augmented textual inversion and a dual prior supervision mechanism. The former enables plug-and-play guidance of stable diffusion for video editing, effectively capturing target attributes for more stylized results. The dual prior supervision mechanism significantly enhances video stability and fidelity. Comprehensive evaluations demonstrate that our approach generates more stable and lifelike videos, outperforming state-of-the-art methods.

📄 PDF Abstract BibTeX arXiv:2410.12526

Code (0)

등록된 구현이 없습니다.

Tasks

Video EditingWord Embeddings

Methods 이 논문이 사용한 방법론

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…
Attention 설명 없음
Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

Parametric Reshaping of Portraits in Videos

2022-05-05 · Xiangjun Tang, Wenxin Sun, Yong-Liang Yang, Xiaogang Jin

Sharing short personalized videos to various social media networks has become quite popular in recent years. This raises the need for digital retouching of portraits in videos. However, applying portrait image editing di…

Face ReconstructionVideo Editing

Audio-driven Talking Face Generation with Stabilized Synchronization Loss

2023-07-18 · Dogucan Yaman, Fevziye Irem Eyiokur, Leonard Bärmann, Hazim Kemal Ekenel 외

Talking face generation aims to create realistic videos with accurate lip synchronization and high visual quality, using given audio and reference video while preserving identity and visual characteristics. In this paper…

Audio-Visual SynchronizationFace GenerationTalking Face Generation

Can We See Photosynthesis? Magnifying the Tiny Color Changes of Plant Green Leaves Using Eulerian Video Magnification

2017-06-12 · Islam A. T. F. Taj-Eddin, Mahmoud Afifi, Mostafa Korashy, Ali H. Ahmed 외

Plant aliveness is proven through laboratory experiments and special scientific instruments. In this paper, we aim to detect the degree of animation of plants based on the magnification of the small color changes in the …

Malicious and Unintentional Disclosure Risks in Large Language Models for Code Generation

2025-03-27 · Rafiqul Rabin, Sean McGregor, Nick Judd

This paper explores the risk that a large language model (LLM) trained for code generation on data mined from software repositories will generate content that discloses sensitive information included in its training data…

Code GenerationLanguage ModelingLanguage ModellingLarge Language Model+1

SoK: Unintended Interactions among Machine Learning Defenses and Risks

2023-12-07 · Vasisht Duddu, Sebastian Szyller, N. Asokan

Machine learning (ML) models cannot neglect risks to security, privacy, and fairness. Several defenses have been proposed to mitigate such risks. When a defense is effective in mitigating one risk, it may correspond to i…

FairnessMemorization