Past Movements-Guided Motion Representation Learning for Human Motion Prediction

2024-08-04 · Junyu Shi, Baoxuan Wang

Human motion prediction based on 3D skeleton is a significant challenge in computer vision, primarily focusing on the effective representation of motion. In this paper, we propose a self-supervised learning framework designed to enhance motion representation. This framework consists of two stages: first, the network is pretrained through the self-reconstruction of past sequences, and the guided reconstruction of future sequences based on past movements. We design a velocity-based mask strategy to focus on the joints with large-scale moving. Subsequently, the pretrained network undergoes finetuning for specific tasks. Self-reconstruction, guided by patterns of past motion, substantially improves the model's ability to represent the spatiotemporal relationships among joints but also captures the latent relationships between past and future sequences. This capability is crucial for motion prediction tasks that solely depend on historical motion data. By employing this straightforward yet effective training paradigm, our method outperforms existing \textit{state-of-the-art} methods, reducing the average prediction errors by 8.8\% across Human3.6M, 3DPW, and AMASS datasets. The code is available at https://github.com/JunyuShi02/PMG-MRL.

📄 PDF Abstract BibTeX arXiv:2408.02091

Code (0)

등록된 구현이 없습니다.

Tasks

Human motion predictionmotion predictionPrediction Representation Learning Self-Supervised Learning

Methods 이 논문이 사용한 방법론

Focus 설명 없음

GazeMotion: Gaze-guided Human Motion Forecasting

2024-03-14 · Zhiming Hu, Syn Schmitt, Daniel Haeufle, Andreas Bulling

We present GazeMotion, a novel method for human motion forecasting that combines information on past human poses with human eye gaze. Inspired by evidence from behavioural sciences showing that human eye and body movemen…

Motion Forecasting

GazeMoDiff: Gaze-guided Diffusion Model for Stochastic Human Motion Prediction

2023-12-19 · Haodong Yan, Zhiming Hu, Syn Schmitt, Andreas Bulling

Human motion prediction is important for many virtual and augmented reality (VR/AR) applications such as collision avoidance and realistic avatar generation. Existing methods have synthesised body motion only from observ…

Collision Avoidance Denoising Graph Attention Human motion prediction+2

Context-aware Human Motion Prediction

2019-04-06 · CVPR 2020 6 · Enric Corona, Albert Pumarola, Guillem Alenyà, Francesc Moreno-Noguer

The problem of predicting human motion given a sequence of past observations is at the core of many applications in robotics and computer vision. Current state-of-the-art formulate this problem as a sequence-to-sequence …

Graph Attention Human motion prediction motion prediction Prediction

Deep representation learning for human motion prediction and classification

2017-02-24 · CVPR 2017 7 · Judith Bütepage, Michael Black, Danica Kragic, Hedvig Kjellström

Generative models of 3D human motion are often restricted to a small number of activities and can therefore not generalize well to novel movements or applications. In this work we propose a deep learning framework for hu…

Action Classification Classification General Classification Human motion prediction+3

Anticipating many futures: Online human motion prediction and synthesis for human-robot collaboration

2017-02-27 · Judith Bütepage, Hedvig Kjellström, Danica Kragic

Fluent and safe interactions of humans and robots require both partners to anticipate the others' actions. A common approach to human intention inference is to model specific trajectories towards known goals with supervi…

Human motion prediction motion prediction