Learning to Generate Novel Scene Compositions from Single Images and Videos
Training GANs in low-data regimes remains a challenge, as overfitting often leads to memorization or training divergence. In this work, we introduce One-Shot GAN that can learn to generate samples from a training set as little as one image or one video. We propose a two-branch discriminator, with content and layout branches designed to judge the internal content separately from the scene layout realism. This allows synthesis of visually plausible, novel compositions of a scene, with varying content and layout, while preserving the context of the original sample. Compared to previous single-image GAN models, One-Shot GAN achieves higher diversity and quality of synthesis. It is also not restricted to the single image setting, successfully learning in the introduced setting of a single video.
Code (1)
Tasks
DiversityMemorizationSimilar Papers 제목 키워드 기반
Generating Novel Scene Compositions from Single Images and Videos
Given a large dataset for training, generative adversarial networks (GANs) can achieve remarkable performance for the image synthesis task. However, training GANs in extremely low data regimes remains a challenge, as ove…
DiversityImage GenerationMemorizationZero-Shot Action Recognition from Diverse Object-Scene Compositions
This paper investigates the problem of zero-shot action recognition, in the setting where no training videos with seen actions are available. For this challenging scenario, the current leading approach is to transfer kno…
Action RecognitionObjectTransfer LearningZero-Shot Action RecognitionGenerating Videos of Zero-Shot Compositions of Actions and Objects
Human activity videos involve rich, varied interactions between people and objects. In this paper we develop methods for generating such videos -- making progress toward addressing the important, open problem of video ge…
Human-Object Interaction DetectionObjectVideo GenerationVideoHandles: Editing 3D Object Compositions in Videos Using Video Generative Priors
Generative methods for image and video editing use generative models as priors to perform edits despite incomplete information, such as changing the composition of 3D objects shown in a single image. Recent methods have …
3D ReconstructionObjectVideo EditingScene Designer: a Unified Model for Scene Search and Synthesis from Sketch
Scene Designer is a novel method for searching and generating images using free-hand sketches of scene compositions; i.e. drawings that describe both the appearance and relative positions of objects. Our core contributio…
Contrastive LearningGraph Neural NetworkObject