Walking on Thin Air: Environment-Free Physics-based Markerless Motion Capture
We propose a generative approach to physics-based motion capture. Unlike prior attempts to incorporate physics into tracking that assume the subject and scene geometry are calibrated and known a priori, our approach is automatic and online. This distinction is important since calibration of the environment is often difficult, especially for motions with props, uneven surfaces, or outdoor scenes. The use of physics in this context provides a natural framework to reason about contact and the plausibility of recovered motions. We propose a fast data-driven parametric body model, based on linear-blend skinning, which decouples deformations due to pose, anthropometrics and body shape. Pose (and shape) parameters are estimated using robust ICP optimization with physics-based dynamic priors that incorporate contact. Contact is estimated from torque trajectories and predictions of which contact points were active. To our knowledge, this is the first approach to take physics into account without explicit {\em a priori} knowledge of the environment or body dimensions. We demonstrate effective tracking from a noisy single depth camera, improving on state-of-the-art results quantitatively and producing better qualitative results, reducing visual artifacts like foot-skate and jitter.
Code (0)
등록된 구현이 없습니다.
Tasks
Markerless Motion CaptureSimilar Papers 제목 키워드 기반
Simplified markerless stride detection pipeline (sMaSDP) for surface EMG segmentation
People with mobility impairments are often recommended for gait assessment studies to diagnose their condition and to select appropriate physiotherapy to improve their mobility. These studies are often conducted in clini…
Electromyography (EMG)Event DetectionSimulating Robotic Locomotion in Sand: Resistive Force Theory in an Open-Source Physics Engine
Recent advancements in Resistive Force Theory (RFT) enable approximation of ground reaction forces for locomotion in sand without the computational expense of modeling interactions with individual grains. However, these …
Annolid: Annotate, Segment, and Track Anything You Need
Annolid is a deep learning-based software package designed for the segmentation, labeling, and tracking of research targets within video files, focusing primarily on animal behavior analysis. Based on state-of-the-art in…
Instance SegmentationSegmentationSemantic SegmentationVideo Object Segmentation+1Differentiable Biomechanics Unlocks Opportunities for Markerless Motion Capture
Recent developments have created differentiable physics simulators designed for machine learning pipelines that can be accelerated on a GPU. While these can simulate biomechanical models, these opportunities have not bee…
GPUMarkerless Motion CaptureWearable-informed generative digital avatars predict task-conditioned post-stroke locomotion
Dynamic prediction of locomotor capacity after stroke could enable more individualized rehabilitation, yet current assessments largely provide static impairment scores and do not indicate whether patients can perform spe…