SignMAE: Segmentation-Driven Self-Supervised Learning for Sign Language Recognition
Subtle hand differences make sign language recognition challenging, yet many existing methods rely on encoders pretrained on generic action datasets that poorly capture such fine-grained cues. We propose a self-supervised pretraining method for sign language recognition that uses segmentation-based masking to adapt to the presence and motion of key body parts, rather than treating hand poses as static visual tokens. The resulting mask-and-reconstruct objective improves fine-grained sign representation learning. On WLASL, NMFs-CSL, and Slovo, our encoder achieves state-of-the-art performance, improving per-instance and per-class Top-1 accuracy while using fewer input frames and modalities than comparable encoders.
Code (0)
등록된 구현이 없습니다.
Tasks
Sign Language RecognitionSelf-Supervised LearningRepresentation LearningSimilar Papers 제목 키워드 기반
Self2Seg: Single-Image Self-Supervised Joint Segmentation and Denoising
We develop Self2Seg, a self-supervised method for the joint segmentation and denoising of a single image. To this end, we combine the advantages of variational segmentation with self-supervised deep learning. One major b…
DenoisingImage DenoisingSegmentationDatUS^2: Data-driven Unsupervised Semantic Segmentation with Pre-trained Self-supervised Vision Transformer
Successive proposals of several self-supervised training schemes continue to emerge, taking one step closer to developing a universal foundation model. In this process, the unsupervised downstream tasks are recognized as…
SegmentationSemantic SegmentationUnsupervised Semantic SegmentationCLIP-S$^4$: Language-Guided Self-Supervised Semantic Segmentation
Existing semantic segmentation approaches are often limited by costly pixel-wise annotations and predefined classes. In this work, we present CLIP-S$^4$ that leverages self-supervised pixel representation learning and vi…
Contrastive LearningLanguage ModellingRepresentation LearningSegmentation+2CLIP-S4: Language-Guided Self-Supervised Semantic Segmentation
Existing semantic segmentation approaches are often limited by costly pixel-wise annotations and predefined classes. In this work, we present CLIP-S^4 that leverages self-supervised pixel representation learning and …
Contrastive LearningLanguage ModellingRepresentation LearningSegmentation+2A Simple Framework for Text-Supervised Semantic Segmentation
Text-supervised semantic segmentation is a novel research topic that allows semantic segments to emerge with image-text contrasting. However, pioneering methods could be subject to specifically designed network archi…
SegmentationSemantic Segmentation