paper-with-me

홈 › Papers

Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videos

2022-09-20 · CVPR 2023 1 · Yilin Wen, Hao Pan, Lei Yang, Jia Pan, Taku Komura, Wenping Wang

Understanding dynamic hand motions and actions from egocentric RGB videos is a fundamental yet challenging task due to self-occlusion and ambiguity. To address occlusion and ambiguity, we develop a transformer-based framework to exploit temporal information for robust estimation. Noticing the different temporal granularity of and the semantic correlation between hand pose estimation and action recognition, we build a network hierarchy with two cascaded transformer encoders, where the first one exploits the short-term temporal cue for hand pose estimation, and the latter aggregates per-frame pose and object information over a longer time span to recognize the action. Our approach achieves competitive results on two first-person hand action benchmarks, namely FPHA and H2O. Extensive ablation studies verify our design choices.

📄 PDF Abstract BibTeX arXiv:2209.09484

Code (1)

fylwen/htt 공식 구현 pytorch

Tasks

3D Hand Pose EstimationAction RecognitionHand Pose EstimationPose Estimation

Similar Papers 제목 키워드 기반

Dual-branch Attention-In-Attention Transformer for single-channel speech enhancement

2021-10-13 · Guochen Yu, Andong Li, Chengshi Zheng, Yinuo Guo 외

Curriculum learning begins to thrive in the speech enhancement area, which decouples the original spectrum estimation task into multiple easier sub-tasks to achieve better performance. Motivated by that, we propose a dua…

Speech Enhancement

HSTFormer: Hierarchical Spatial-Temporal Transformers for 3D Human Pose Estimation

2023-01-18 · Xiaoye Qian, YouBao Tang, Ning Zhang, Mei Han 외

Transformer-based approaches have been successfully proposed for 3D human pose estimation (HPE) from 2D pose sequence and achieved state-of-the-art (SOTA) performance. However, current SOTAs have difficulties in modeling…

3D Human Pose EstimationPose Estimation

Disentangled Diffusion-Based 3D Human Pose Estimation with Hierarchical Spatial and Temporal Denoiser

2024-03-07 · Qingyuan Cai, Xuecai Hu, Saihui Hou, Li Yao 외

Recently, diffusion-based methods for monocular 3D human pose estimation have achieved state-of-the-art (SOTA) performance by directly regressing the 3D joint coordinates from the 2D pose sequence. Although some methods …

3D Human Pose EstimationDisentanglementMonocular 3D Human Pose EstimationMulti-Hypotheses 3D Human Pose Estimation+1

Hierarchical Spatiotemporal Transformers for Video Object Segmentation

2023-07-17 · Jun-Sang Yoo, Hongjae Lee, Seung-Won Jung

This paper presents a novel framework called HST for semi-supervised video object segmentation (VOS). HST extracts image and video features using the latest Swin Transformer and Video Swin Transformer to inherit their in…

Inductive BiasObjectSemantic SegmentationSemi-Supervised Video Object Segmentation+2

H$_{2}$OT: Hierarchical Hourglass Tokenizer for Efficient Video Pose Transformers

2025-09-08 · Wenhao Li, Mengyuan Liu, Hong Liu, Pichao Wang 외 arxiv

Transformers have been successfully applied in the field of video-based 3D human pose estimation. However, the high computational costs of these video pose transformers (VPTs) make them impractical on resource-constraine…

3D Human Pose Estimation