A comparison of small sample methods for Handshape Recognition
Automatic Sign Language Translation (SLT) systems can be a great asset to improve the communication with and within deaf communities. Currently, the main issue preventing effective translation models lays in the low availability of labelled data, which hinders the use of modern deep learning models. SLT is a complex problem that involves many subtasks, of which handshape recognition is the most important. We compare a series of models specially tailored for small datasets to improve their performance on handshape recognition tasks. We evaluate Wide-DenseNet and few-shot Prototypical Network models with and without transfer learning, and also using Model-Agnostic Meta-Learning (MAML). Our findings indicate that Wide-DenseNet without transfer learning and Prototipical Networks with transfer learning provide the best results. Prototypical networks, particularly, are vastly superior when using less than 30 samples, while Wide-DenseNet achieves the best results with more samples. On the other hand, MAML does not improve performance in any scenario. These results can help to design better SLT models.
Code (1)
Tasks
Hand Gesture RecognitionMeta-LearningSign Language TranslationTransfer LearningTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Improving Handshape Representations for Sign Language Processing: A Graph Neural Network Approach
Handshapes serve a fundamental phonological role in signed languages, with American Sign Language employing approximately 50 distinct shapes. However,computational approaches rarely model handshapes explicitly, limiting …
Graph Neural NetworkContrastive LearningMHB: Multimodal Handshape-aware Boundary Detection for Continuous Sign Language Recognition
This paper employs a multimodal approach for continuous sign recognition by first using ML for detecting the start and end frames of signs in videos of American Sign Language (ASL) sentences, and then by recognizing the …
Sign Language RecognitionBoundary DetectionVideo SegmentationA Study of Convolutional Architectures for Handshape Recognition applied to Sign Language
Convolutional Neural Networks have been providing a performance boost in many areas in the last few years, but their performance for Handshape Recognition in the context of Sign Language Recognition has not been thorough…
Hand Gesture RecognitionSign Language RecognitionTransfer LearningHandshape recognition for Argentinian Sign Language using ProbSom
Automatic sign language recognition is an important topic within the areas of human-computer interaction and machine learning. On the one hand, it poses a complex challenge that requires the intervention of various knowl…
Sign Language RecognitionBringing Balance to Hand Shape Classification: Mitigating Data Imbalance Through Generative Models
Most sign language handshape datasets are severely limited and unbalanced, posing significant challenges to effective model training. In this paper, we explore the effectiveness of augmenting the training data of a hands…