Learning from What is Already Out There: Few-shot Sign Language Recognition with Online Dictionaries
Today's sign language recognition models require large training corpora of laboratory-like videos, whose collection involves an extensive workforce and financial resources. As a result, only a handful of such systems are publicly available, not to mention their limited localization capabilities for less-populated sign languages. Utilizing online text-to-video dictionaries, which inherently hold annotated data of various attributes and sign languages, and training models in a few-shot fashion hence poses a promising path for the democratization of this technology. In this work, we collect and open-source the UWB-SL-Wild few-shot dataset, the first of its kind training resource consisting of dictionary-scraped videos. This dataset represents the actual distribution and characteristics of available online sign language data. We select glosses that directly overlap with the already existing datasets WLASL100 and ASLLVD and share their class mappings to allow for transfer learning experiments. Apart from providing baseline results on a pose-based architecture, we introduce a novel approach to training sign language recognition models in a few-shot scenario, resulting in state-of-the-art results on ASLLVD-Skeleton and ASLLVD-Skeleton-20 datasets with top-1 accuracy of $30.97~\%$ and $95.45~\%$, respectively.
Code (1)
Tasks
Sign Language RecognitionTransfer LearningSimilar Papers 제목 키워드 기반
Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphs
In any system that uses structured knowledge graph (KG) data as its underlying knowledge representation, KG-to-text generation is a useful tool for turning parts of the graph data into text that can be understood by huma…
KG-to-Text GenerationKnowledge GraphsText GenerationA Training-Free Guess What Vision Language Model from Snippets to Open-Vocabulary Object Detection
Open-Vocabulary Object Detection (OVOD) aims to develop the capability to detect anything. Although myriads of large-scale pre-training efforts have built versatile foundation models that exhibit impressive zero-shot cap…
Object DetectionThe Score Granularity Gap in Black-Box LLM Classification: A Comparative Study of Confidence Constructions
Large language models (LLMs) are increasingly deployed as black-box classifiers in pipelines that automate confident decisions and route uncertain ones to human review. Such selective prediction needs a confidence score …
DNAHLM -- DNA sequence and Human Language mixed large language Model
There are already many DNA large language models, but most of them still follow traditional uses, such as extracting sequence features for classification tasks. More innovative applications of large language models, such…
Language ModelingLanguage ModellingLarge Language ModelPrompt Engineering+1Zero-shot Reading Comprehension by Cross-lingual Transfer Learning with Multi-lingual Language Representation Model
Because it is not feasible to collect training data for every language, there is a growing interest in cross-lingual transfer learning. In this paper, we systematically explore zero-shot cross-lingual transfer learning o…
Cross-Lingual TransferReading ComprehensionTransfer LearningZero-Shot Cross-Lingual Transfer+1