Skeleton-Graph: Long-Term 3D Motion Prediction From 2D Observations Using Deep Spatio-Temporal Graph CNNs

2021-09-21 · Abduallah Mohamed, Huancheng Chen, Zhangyang Wang, Christian Claudel

Several applications such as autonomous driving, augmented reality and virtual reality require a precise prediction of the 3D human pose. Recently, a new problem was introduced in the field to predict the 3D human poses from observed 2D poses. We propose Skeleton-Graph, a deep spatio-temporal graph CNN model that predicts the future 3D skeleton poses in a single pass from the 2D ones. Unlike prior works, Skeleton-Graph focuses on modeling the interaction between the skeleton joints by exploiting their spatial configuration. This is being achieved by formulating the problem as a graph structure while learning a suitable graph adjacency kernel. By the design, Skeleton-Graph predicts the future 3D poses without divergence in the long-term, unlike prior works. We also introduce a new metric that measures the divergence of predictions in the long term. Our results show an FDE improvement of at least 27% and an ADE of 4% on both the GTA-IM and PROX datasets respectively in comparison with prior works. Also, we are 88% and 93% less divergence on the long-term motion prediction in comparison with prior works on both GTA-IM and PROX datasets. Code is available at https://github.com/abduallahmohamed/Skeleton-Graph.git

📄 PDF Abstract BibTeX arXiv:2109.10257

Code (1)

abduallahmohamed/skeleton-graph 공식 구현 pytorch

Tasks

Autonomous Drivingmotion predictionTrajectory Prediction

Dynamic Dense Graph Convolutional Network for Skeleton-based Human Motion Prediction

2023-11-29 · Xinshun Wang, Wanying Zhang, Can Wang, Yuan Gao 외

Graph Convolutional Networks (GCN) which typically follows a neural message passing framework to model dependencies among skeletal joints has achieved high success in skeleton-based human motion prediction task. Neverthe…

Human motion prediction motion prediction

Multiscale Spatio-Temporal Graph Neural Networks for 3D Skeleton-Based Motion Prediction

2021-08-25 · Maosen Li, Siheng Chen, Yangheng Zhao, Ya zhang 외

We propose a multiscale spatio-temporal graph neural network (MST-GNN) to predict the future 3D skeleton-based human poses in an action-category-agnostic manner. The core of MST-GNN is a multiscale spatio-temporal graph …

Decoder Graph Neural Network motion prediction

The MI-Motion Dataset and Benchmark for 3D Multi-Person Motion Prediction

2023-06-23 · Xiaogang Peng, Xiao Zhou, Yikai Luo, Hao Wen 외

3D multi-person motion prediction is a challenging task that involves modeling individual behaviors and interactions between people. Despite the emergence of approaches for this task, comparing them is difficult due to t…

motion prediction Prediction

Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction

2020-03-17 · Maosen Li, Siheng Chen, Yangheng Zhao, Ya zhang 외

We propose novel dynamic multiscale graph neural networks (DMGNN) to predict 3D skeleton-based human motions. The core idea of DMGNN is to use a multiscale graph to comprehensively model the internal relations of a human…

3D Human Pose Estimation 3D Pose Estimation Decoder Human motion prediction+1

Dynamic Multiscale Graph Neural Networks for 3D Skeleton Based Human Motion Prediction

2020-06-01 · CVPR 2020 6 · Maosen Li, Siheng Chen, Yangheng Zhao, Ya Zhang 외

Decoder Human motion prediction motion prediction