How Animals Dance (When You're Not Looking)

2025-05-29 · Xiaojuan Wang, Aleksander Holynski, Brian Curless, Ira Kemelmacher, Steve Seitz

We present a keyframe-based framework for generating music-synchronized, choreography aware animal dance videos. Starting from a few keyframes representing distinct animal poses -- generated via text-to-image prompting or GPT-4o -- we formulate dance synthesis as a graph optimization problem: find the optimal keyframe structure that satisfies a specified choreography pattern of beats, which can be automatically estimated from a reference dance video. We also introduce an approach for mirrored pose image generation, essential for capturing symmetry in dance. In-between frames are synthesized using an video diffusion model. With as few as six input keyframes, our method can produce up to 30 second dance videos across a wide range of animals and music tracks.

📄 PDF Abstract BibTeX arXiv:2505.23738

Code (0)

등록된 구현이 없습니다.

Tasks

Image Generation

Methods 이 논문이 사용한 방법론

Diffusion Diffusion models generate samples by gradually removing noise from a signal, and their training objective can be expressed as a reweighted variational lower-bound…

AWARE We propose to theoretically and empirically examine the effect of incorporating weighting schemes into walk-aggregating GNNs. To this end, we propose a simple, interpretable, and…

Unsupervisedly Learned Representations: Should the Quest be Over?

2020-01-21 · Daniel N. Nissani

After four decades of research there still exists a Classification accuracy gap of about 20% between our best Unsupervisedly Learned Representations methods and the accuracy rates achieved by intelligent animals. It thus…

General Classification reinforcement-learning Reinforcement Learning Reinforcement Learning (RL)

Hypergraph Animals

2023-04-03 · Michael P. H. Stumpf

Here we introduce simple structures for the analysis of complex hypergraphs, hypergraph animals. These structures are designed to describe the local node neighbourhoods of nodes in hypergraphs. We establish their relatio…

Multi-animal pose estimation, identification and tracking with DeepLabCut

2022-04-12 · Nature Methods 2022 4 · Jessy Lauer, Mu Zhou, Shaokai Ye, William Menegas 외

Estimating the pose of multiple animals is a challenging computer vision problem: frequent interactions cause occlusions and complicate the association of detected keypoints to the correct individuals, as well as having …

Animal Pose Estimation Pose Estimation

What can I do here? A Theory of Affordances in Reinforcement Learning

2020-06-26 · ICML 2020 1 · Khimya Khetarpal, Zafarali Ahmed, Gheorghe Comanici, David Abel 외

Reinforcement learning algorithms usually assume that all actions are always available to an agent. However, both people and animals understand the general link between the features of their environment and the actions t…

reinforcement-learning Reinforcement Learning (RL)

Emotion Recognition in Horses with Convolutional Neural Networks

2021-05-25 · Luis A. Corujo, Peter A. Gloor, Emily Kieson, Timo Schloesser

Creating intelligent systems capable of recognizing emotions is a difficult task, especially when looking at emotions in animals. This paper describes the process of designing a "proof of concept" system to recognize emo…

Emotion Recognition