Semantics-aware Test-time Adaptation for 3D Human Pose Estimation
This work highlights a semantics misalignment in 3D human pose estimation. For the task of test-time adaptation, the misalignment manifests as overly smoothed and unguided predictions. The smoothing settles predictions towards some average pose. Furthermore, when there are occlusions or truncations, the adaptation becomes fully unguided. To this end, we pioneer the integration of a semantics-aware motion prior for the test-time adaptation of 3D pose estimation. We leverage video understanding and a well-structured motion-text space to adapt the model motion prediction to adhere to video semantics during test time. Additionally, we incorporate a missing 2D pose completion based on the motion-text similarity. The pose completion strengthens the motion prior's guidance for occlusions and truncations. Our method significantly improves state-of-the-art 3D human pose estimation TTA techniques, with more than 12% decrease in PA-MPJPE on 3DPW and 3DHP.
Code (0)
등록된 구현이 없습니다.
Tasks
3D human pose and shape estimation3D Human Pose Estimation3D Pose Estimationmotion predictionPose EstimationTest-time Adaptationtext similarityVideo UnderstandingSimilar Papers 제목 키워드 기반
Temporal Memory-Aware Online Test-Time Adaptation on Dynamic Graphs
Test-time adaptation (TTA) on graphs aims to adapt a graph neural network (GNN) that is well-trained on the training graph to the test graph, which involves potential distribution shifts that may harm model generalizatio…
Test-time AdaptationGraph Neural NetworkCTTA-T: Continual Test-Time Adaptation for Text Understanding via Teacher-Student with a Domain-aware and Generalized Teacher
Text understanding often suffers from domain shifts. To handle testing domains, domain adaptation (DA) is trained to adapt to a fixed and observed testing domain; a more challenging paradigm, test-time adaptation (TTA), …
Test-time AdaptationDomain AdaptationFast Adaptation for Human Pose Estimation via Meta-Optimization
Domain shift is a challenge for supervised human pose estimation where the source data and target data come from different distributions. This is why pose estimation methods generally perform worse on the test set th…
Auxiliary LearningImage InpaintingMeta-LearningPose Estimation+2Majorization-Guided Test-Time Adaptation for Vision-Language Models under Modality-Specific Shift
Vision-language models transfer well in zero-shot settings, but at deployment the visual and textual branches often shift asymmetrically. Under this condition, entropy-based test-time adaptation can sharpen the fused pos…
Test-time AdaptationSAM-aware Test-time Adaptation for Universal Medical Image Segmentation
Universal medical image segmentation using the Segment Anything Model (SAM) remains challenging due to its limited adaptability to medical domains. Existing adaptations, such as MedSAM, enhance SAM's performance in medic…
Image SegmentationMedical Image SegmentationSemantic SegmentationTest-time Adaptation