paper-with-me

Papers

MarioNette: Self-Supervised Sprite Learning

2021-04-29 · NeurIPS 2021 12 · Dmitriy Smirnov, Michael Gharbi, Matthew Fisher, Vitor Guizilini, Alexei A. Efros, Justin Solomon

Artists and video game designers often construct 2D animations using libraries of sprites -- textured patches of objects and characters. We propose a deep learning approach that decomposes sprite-based video animations into a disentangled representation of recurring graphic elements in a self-supervised manner. By jointly learning a dictionary of possibly transparent patches and training a network that places them onto a canvas, we deconstruct sprite-based content into a sparse, consistent, and explicit representation that can be easily used in downstream tasks, like editing or analysis. Our framework offers a promising approach for discovering recurring visual patterns in image collections without supervision.

📄 PDF Abstract BibTeX arXiv:2104.14553

Code (1)

dmsm/MarioNette 공식 구현 pytorch

Similar Papers 제목 키워드 기반

Neural Marionette: Unsupervised Learning of Motion Skeleton and Latent Dynamics from Volumetric Video

2022-02-17 · Jinseok Bae, Hojun Jang, Cheol-Hui Min, Hyungun Choi 외

We present Neural Marionette, an unsupervised approach that discovers the skeletal structure from a dynamic sequence and learns to generate diverse motions that are consistent with the observed motion dynamics. Given a v…

motion retargeting

Self-Supervised Equivariant Scene Synthesis from Video

2021-02-01 · Cinjon Resnick, Or Litany, Cosmas Heiß, Hugo Larochelle 외

We propose a self-supervised framework to learn scene representations from video that are automatically delineated into background, characters, and their animations. Our method capitalizes on moving characters being equi…

Deformable Sprites for Unsupervised Video Decomposition

2022-04-14 · CVPR 2022 1 · Vickie Ye, Zhengqi Li, Richard Tucker, Angjoo Kanazawa 외

We describe a method to extract persistent elements of a dynamic scene from an input video. We represent each scene element as a \emph{Deformable Sprite} consisting of three components: 1) a 2D texture image for the enti…

Video Editing

Unsupervised Layered Image Decomposition into Object Prototypes

2021-04-29 · ICCV 2021 10 · Tom Monnier, Elliot Vincent, Jean Ponce, Mathieu Aubry

We present an unsupervised learning framework for decomposing images into layers of automatically discovered object models. Contrary to recent approaches that model image layers with autoencoder networks, we represent th…

ObjectObject Discovery

Motion Marionette: Rethinking Rigid Motion Transfer via Prior Guidance

2025-11-25 · Haoxuan Wang, Jiachen Tao, Junyi Wu, Gaowen Liu 외 arxiv

We present Motion Marionette, a zero-shot framework for rigid motion transfer from monocular source videos to single-view target images. Previous works typically employ geometric, generative, or simulation priors to guid…

Video Generation