Improving Real-Time Omnidirectional 3D Multi-Person Human Pose Estimation with People Matching and Unsupervised 2D-3D Lifting
Current human pose estimation systems focus on retrieving an accurate 3D global estimate of a single person. Therefore, this paper presents one of the first 3D multi-person human pose estimation systems that is able to work in real-time and is also able to handle basic forms of occlusion. First, we adjust an off-the-shelf 2D detector and an unsupervised 2D-3D lifting model for use with a 360$^\circ$ panoramic camera and mmWave radar sensors. We then introduce several contributions, including camera and radar calibrations, and the improved matching of people within the image and radar space. The system addresses both the depth and scale ambiguity problems by employing a lightweight 2D-3D pose lifting algorithm that is able to work in real-time while exhibiting accurate performance in both indoor and outdoor environments which offers both an affordable and scalable solution. Notably, our system's time complexity remains nearly constant irrespective of the number of detected individuals, achieving a frame rate of approximately 7-8 fps on a laptop with a commercial-grade GPU.
Code (0)
등록된 구현이 없습니다.
Tasks
3D Multi-Person Human Pose EstimationGPUPose EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Human Pose Estimation in Monocular Omnidirectional Top-View Images
Human pose estimation (HPE) with convolutional neural networks (CNNs) for indoor monitoring is one of the major challenges in computer vision. In contrast to HPE in perspective views, an indoor monitoring system can cons…
2D Human Pose Estimation3D Human Pose EstimationKeypoint DetectionPose EstimationOmniPD: One-Step Person Detection in Top-View Omnidirectional Indoor Scenes
We propose a one-step person detector for topview omnidirectional indoor scenes based on convolutional neural networks (CNNs). While state of the art person detectors reach competitive results on perspective images, miss…
Data AugmentationHuman DetectionTransfer LearningRL-DWA Omnidirectional Motion Planning for Person Following in Domestic Assistance and Monitoring
Robot assistants are emerging as high-tech solutions to support people in everyday life. Following and assisting the user in the domestic environment requires flexible mobility to safely move in cluttered spaces. We intr…
Deep Reinforcement LearningMotion PlanningNavigateWeakly-Supervised Multi-Person Action Recognition in 360$^{\circ}$ Videos
The recent development of commodity 360$^{\circ}$ cameras have enabled a single video to capture an entire scene, which endows promising potentials in surveillance scenarios. However, research in omnidirectional video an…
Action LocalizationAction RecognitionMulti-Label LearningBridge the Gap Between VQA and Human Behavior on Omnidirectional Video: A Large-Scale Dataset and a Deep Learning Model
Omnidirectional video enables spherical stimuli with the $360 \times 180^ \circ$ viewing range. Meanwhile, only the viewport region of omnidirectional video can be seen by the observer through head movement (HM), and an …
Visual Question Answering (VQA)