Stochastic Multi-Person 3D Motion Forecasting
This paper aims to deal with the ignored real-world complexities in prior work on human motion forecasting, emphasizing the social properties of multi-person motion, the diversity of motion and social interactions, and the complexity of articulated motion. To this end, we introduce a novel task of stochastic multi-person 3D motion forecasting. We propose a dual-level generative modeling framework that separately models independent individual motion at the local level and social interactions at the global level. Notably, this dual-level modeling mechanism can be achieved within a shared generative model, through introducing learnable latent codes that represent intents of future motion and switching the codes' modes of operation at different levels. Our framework is general; we instantiate it with different generative models, including generative adversarial networks and diffusion models, and various multi-person forecasting models. Extensive experiments on CMU-Mocap, MuPoTS-3D, and SoMoF benchmarks show that our approach produces diverse and accurate multi-person predictions, significantly outperforming the state of the art.
Code (1)
Tasks
DiversityHuman motion predictionMotion ForecastingStochastic Human Motion PredictionMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Massively Multi-Person 3D Human Motion Forecasting with Scene Context
Forecasting long-term 3D human motion is challenging: the stochasticity of human behavior makes it hard to generate realistic human motion from the input sequence alone. Information on the scene environment and the motio…
DecoderDenoisingDiversityMotion ForecastingSnipper: A Spatiotemporal Transformer for Simultaneous Multi-Person 3D Pose Estimation Tracking and Forecasting on a Video Snippet
Multi-person pose understanding from RGB videos involves three complex tasks: pose estimation, tracking and motion forecasting. Intuitively, accurate multi-person pose estimation facilitates robust tracking, and robust t…
3D Pose EstimationMotion ForecastingMulti-Person Pose EstimationPose EstimationSoMoFormer: Multi-Person Pose Forecasting with Transformers
Human pose forecasting is a challenging problem involving complex human body motion and posture dynamics. In cases that there are multiple people in the environment, one's motion may also be influenced by the motion and …
Human Pose Forecastingmotion predictionMulti-Person Pose forecastingTime Series AnalysisHumans in Kitchens: A Dataset for Multi-Person Human Motion Forecasting with Scene Context
Forecasting human motion of multiple persons is very challenging. It requires to model the interactions between humans and the interactions with objects and the environment. For example, a person might want to make a cof…
Personalized Pose Forecasting
Human pose forecasting is the task of predicting articulated human motion given past human motion. There exists a number of popular benchmarks that evaluate an array of different models performing human pose forecasting.…
Human Pose ForecastingMotion ForecastingTime SeriesTime Series Analysis