paper-with-me

Papers

CamContextI2V: Context-aware Controllable Video Generation

2025-04-08 · Luis Denninger, Sina Mokhtarzadeh Azar, Juergen Gall

Recently, image-to-video (I2V) diffusion models have demonstrated impressive scene understanding and generative quality, incorporating image conditions to guide generation. However, these models primarily animate static images without extending beyond their provided context. Introducing additional constraints, such as camera trajectories, can enhance diversity but often degrades visual quality, limiting their applicability for tasks requiring faithful scene representation. We propose CamContextI2V, an I2V model that integrates multiple image conditions with 3D constraints alongside camera control to enrich both global semantics and fine-grained visual details. This enables more coherent and context-aware video generation. Moreover, we motivate the necessity of temporal awareness for an effective context representation. Our comprehensive study on the RealEstate10K dataset demonstrates improvements in visual quality and camera controllability. We make our code and models publicly available at: https://github.com/LDenninger/CamContextI2V.

📄 PDF Abstract BibTeX arXiv:2504.06022

Code (1)

ldenninger/camcontexti2v 공식 구현 pytorch

Tasks

DiversityScene UnderstandingVideo Generation

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

Similar Papers 제목 키워드 기반

Context-aware Talking Face Video Generation

2024-02-28 · Meidai Xuanyuan, Yuwang Wang, Honglei Guo, Qionghai Dai

In this paper, we consider a novel and practical case for talking face video generation. Specifically, we focus on the scenarios involving multi-people interactions, where the talking context, such as audience or surroun…

Video GenerationVideo Synchronization

Controllable Audio-Visual Viewpoint Generation from 360° Spatial Information

2025-10-07 · Christian Marinoni, Riccardo Fosco Gramaccioni, Eleonora Grassucci, Danilo Comminiello arxiv

The generation of sounding videos has seen significant advancements with the advent of diffusion models. However, existing methods often lack the fine-grained control needed to generate viewpoint-specific content from la…

PE-Field 4D: Video Generation Models as Canvas

2026-07-17 · Yunpeng Bai, Haoxiang Li, Qixing Huang arxiv

Diffusion Transformers have recently achieved strong performance in video generation, yet controlling scene geometry under viewpoint changes and camera motion remains challenging. In this work, we revisit the role of pos…

Video Generation

DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion

2024-11-07 · Wenqiang Sun, Shuo Chen, Fangfu Liu, Zilong Chen 외

In this paper, we introduce \textbf{DimensionX}, a framework designed to generate photorealistic 3D and 4D scenes from just a single image with video diffusion. Our approach begins with the insight that both the spatial …

3D GenerationDenoisingScene GenerationVideo Generation

MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model

2024-05-30 · Muyao Niu, Xiaodong Cun, Xintao Wang, Yong Zhang 외

We present MOFA-Video, an advanced controllable image animation method that generates video from the given image using various additional controllable signals (such as human landmarks reference, manual trajectories, and …

Image AnimationVideo Generation