MarioNette: Self-Supervised Sprite Learning
Artists and video game designers often construct 2D animations using libraries of sprites -- textured patches of objects and characters. We propose a deep learning approach that decomposes sprite-based video animations into a disentangled representation of recurring graphic elements in a self-supervised manner. By jointly learning a dictionary of possibly transparent patches and training a network that places them onto a canvas, we deconstruct sprite-based content into a sparse, consistent, and explicit representation that can be easily used in downstream tasks, like editing or analysis. Our framework offers a promising approach for discovering recurring visual patterns in image collections without supervision.
Code (1)
Similar Papers 제목 키워드 기반
Neural Marionette: Unsupervised Learning of Motion Skeleton and Latent Dynamics from Volumetric Video
We present Neural Marionette, an unsupervised approach that discovers the skeletal structure from a dynamic sequence and learns to generate diverse motions that are consistent with the observed motion dynamics. Given a v…
motion retargetingSelf-Supervised Equivariant Scene Synthesis from Video
We propose a self-supervised framework to learn scene representations from video that are automatically delineated into background, characters, and their animations. Our method capitalizes on moving characters being equi…
Deformable Sprites for Unsupervised Video Decomposition
We describe a method to extract persistent elements of a dynamic scene from an input video. We represent each scene element as a \emph{Deformable Sprite} consisting of three components: 1) a 2D texture image for the enti…
Video EditingUnsupervised Layered Image Decomposition into Object Prototypes
We present an unsupervised learning framework for decomposing images into layers of automatically discovered object models. Contrary to recent approaches that model image layers with autoencoder networks, we represent th…
ObjectObject DiscoveryMotion Marionette: Rethinking Rigid Motion Transfer via Prior Guidance
We present Motion Marionette, a zero-shot framework for rigid motion transfer from monocular source videos to single-view target images. Previous works typically employ geometric, generative, or simulation priors to guid…
Video Generation