Creatures great and SMAL: Recovering the shape and motion of animals from video
We present a system to recover the 3D shape and motion of a wide variety of quadrupeds from video. The system comprises a machine learning front-end which predicts candidate 2D joint positions, a discrete optimization which finds kinematically plausible joint correspondences, and an energy minimization stage which fits a detailed 3D model to the image. In order to overcome the limited availability of motion capture training data from animals, and the difficulty of generating realistic synthetic training images, the system is designed to work on silhouette data. The joint candidate predictor is trained on synthetically generated silhouette images, and at test time, deep learning methods or standard video segmentation tools are used to extract silhouettes from real data. The system is tested on animal videos from several species, and shows accurate reconstructions of 3D shape and pose.
Code (0)
등록된 구현이 없습니다.
Tasks
Video SegmentationVideo Semantic SegmentationSimilar Papers 제목 키워드 기반
Interactive embodied evolution for socially adept Artificial General Creatures
We introduce here the concept of Artificial General Creatures (AGC) which encompasses "robotic or virtual agents with a wide enough range of capabilities to ensure their continued survival". With this in mind, we propose…
EthicsRecovering 3D Shapes from Ultra-Fast Motion-Blurred Images
We consider the problem of 3D shape recovery from ultra-fast motion-blurred images. While 3D reconstruction from static images has been extensively studied, recovering geometry from extreme motion-blurred images remains …
Inverse Rendering3D ReconstructionHuMoR: 3D Human Motion Model for Robust Pose Estimation
We introduce HuMoR: a 3D Human Motion Model for Robust Estimation of temporal pose and shape. Though substantial progress has been made in estimating 3D human motion and shape from dynamic observations, recovering plausi…
Pose EstimationSingle Motion Diffusion
Synthesizing realistic animations of humans, animals, and even imaginary creatures, has long been a goal for artists and computer graphics professionals. Compared to the imaging domain, which is rich with large available…
DenoisingStyle TransferFlow-Lenia: Towards open-ended evolution in cellular automata through mass conservation and parameter localization
The design of complex self-organising systems producing life-like phenomena, such as the open-ended evolution of virtual creatures, is one of the main goals of artificial life. Lenia, a family of cellular automata (CA) g…
Artificial Life