VideoHandles: Editing 3D Object Compositions in Videos Using Video Generative Priors
Generative methods for image and video editing use generative models as priors to perform edits despite incomplete information, such as changing the composition of 3D objects shown in a single image. Recent methods have shown promising composition editing results in the image setting, but in the video setting, editing methods have focused on editing object's appearance and motion, or camera motion, and as a result, methods to edit object composition in videos are still missing. We propose \name as a method for editing 3D object compositions in videos of static scenes with camera motion. Our approach allows editing the 3D position of a 3D object across all frames of a video in a temporally consistent manner. This is achieved by lifting intermediate features of a generative model to a 3D reconstruction that is shared between all frames, editing the reconstruction, and projecting the features on the edited reconstruction back to each frame. To the best of our knowledge, this is the first generative approach to edit object compositions in videos. Our approach is simple and training-free, while outperforming state-of-the-art image editing baselines.
Code (0)
등록된 구현이 없습니다.
Tasks
3D ReconstructionObjectVideo EditingSimilar Papers 제목 키워드 기반
Generative Omnimatte: Learning to Decompose Video into Layers
Given a video and a set of input object masks, an omnimatte method aims to decompose the video into semantically meaningful layers containing individual objects along with their associated effects, such as shadows and re…
Depth EstimationVideo InpaintingZero-Shot Action Recognition from Diverse Object-Scene Compositions
This paper investigates the problem of zero-shot action recognition, in the setting where no training videos with seen actions are available. For this challenging scenario, the current leading approach is to transfer kno…
Action RecognitionObjectTransfer LearningZero-Shot Action RecognitionVCoME: Verbal Video Composition with Multimodal Editing Effects
Verbal videos, featuring voice-overs or text overlays, provide valuable content but present significant challenges in composition, especially when incorporating editing effects to enhance clarity and visual appeal. In th…
Videoshop: Localized Semantic Video Editing with Noise-Extrapolated Diffusion Inversion
We introduce Videoshop, a training-free video editing algorithm for localized semantic edits. Videoshop allows users to use any editing software, including Photoshop and generative inpainting, to modify the first frame; …
Video EditingGenerating Videos of Zero-Shot Compositions of Actions and Objects
Human activity videos involve rich, varied interactions between people and objects. In this paper we develop methods for generating such videos -- making progress toward addressing the important, open problem of video ge…
Human-Object Interaction DetectionObjectVideo Generation