Fusing Wearable IMUs with Multi-View Images for Human Pose Estimation: A Geometric Approach
We propose to estimate 3D human pose from multi-view images and a few IMUs attached at person's limbs. It operates by firstly detecting 2D poses from the two signals, and then lifting them to the 3D space. We present a geometric approach to reinforce the visual features of each pair of joints based on the IMUs. This notably improves 2D pose estimation accuracy especially when one joint is occluded. We call this approach Orientation Regularized Network (ORN). Then we lift the multi-view 2D poses to the 3D space by an Orientation Regularized Pictorial Structure Model (ORPSM) which jointly minimizes the projection error between the 3D and 2D poses, along with the discrepancy between the 3D pose and IMU orientations. The simple two-step approach reduces the error of the state-of-the-art by a large margin on a public dataset. Our code will be released at https://github.com/CHUNYUWANG/imu-human-pose-pytorch.
Code (1)
Tasks
2D Pose Estimation3D Absolute Human Pose Estimation3D Human Pose EstimationPose EstimationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fusing uncalibrated IMUs and handheld smartphone video to reconstruct knee kinematics
Video and wearable sensor data provide complementary information about human movement. Video provides a holistic understanding of the entire body in the world while wearable sensors provide high-resolution measurements o…
Markerless Motion CaptureSensor FusionUMotion: Uncertainty-driven Human Motion Estimation from Inertial and Ultra-wideband Units
Sparse wearable inertial measurement units (IMUs) have gained popularity for estimating 3D human motion. However, challenges such as pose ambiguity, data drift, and limited adaptability to diverse bodies persist. To addr…
Motion EstimationPose EstimationState EstimationAn Examination of Wearable Sensors and Video Data Capture for Human Exercise Classification
Wearable sensors such as Inertial Measurement Units (IMUs) are often used to assess the performance of human exercise. Common approaches use handcrafted features based on domain expertise or automatically extracted featu…
Feature EngineeringTime SeriesTime Series AnalysisSuite-IN: Aggregating Motion Features from Apple Suite for Robust Inertial Navigation
With the rapid development of wearable technology, devices like smartphones, smartwatches, and headphones equipped with IMUs have become essential for applications such as pedestrian positioning. However, traditional ped…
StrengthSense: A Dataset of IMU Signals Capturing Everyday Strength-Demanding Activities
Tracking strength-demanding activities with wearable sensors like IMUs is crucial for monitoring muscular strength, endurance, and power. However, there is a lack of comprehensive datasets capturing these activities. To …
Human Activity Recognition