Papers Gloss-free Sign Language Translation
“Gloss-free Sign Language Translation” 태그가 달린 논문 20편 · 필터 해제
Multilingual Gloss-free Sign Language Translation: Towards Building a Sign Language Foundation Model
Sign Language Translation (SLT) aims to convert sign language (SL) videos into spoken language text, thereby bridging the communication gap between the sign and the spoken community. While most existing works focus on tr…
Gloss-free Sign Language TranslationSign Language TranslationText GenerationUni-Sign: Toward Unified Sign Language Understanding at Scale
Sign language pre-training has gained increasing attention for its ability to enhance performance across various sign language understanding (SLU) tasks. However, existing methods often suffer from a gap between pre-trai…
Computational EfficiencyGloss-free Sign Language TranslationSign Language RecognitionSign Language Translation+1Leveraging the Power of MLLMs for Gloss-Free Sign Language Translation
Sign language translation (SLT) is a challenging task that involves translating sign language images into spoken language. For SLT models to perform this task successfully, they must bridge the modality gap and identify …
Gloss-free Sign Language TranslationSentenceSign Language TranslationTranslationSignformer is all you need: Towards Edge AI for Sign Language
Sign language translation, especially in gloss-free paradigm, is confronting a dilemma of impracticality and unsustainability due to growing resource-intensive methodologies. Contemporary state-of-the-arts (SOTAs) have s…
AllGloss-free Sign Language TranslationSign Language TranslationAn Efficient Sign Language Translation Using Spatial Configuration and Motion Dynamics with LLMs
Gloss-free Sign Language Translation (SLT) converts sign videos directly into spoken language sentences without relying on glosses. Recently, Large Language Models (LLMs) have shown remarkable translation performance in …
Gloss-free Sign Language TranslationSign Language TranslationC${^2}$RL: Content and Context Representation Learning for Gloss-free Sign Language Translation and Retrieval
Sign Language Representation Learning (SLRL) is crucial for a range of sign language-related downstream tasks such as Sign Language Translation (SLT) and Sign Language Retrieval (SLRet). Recently, many gloss-based and gl…
Gloss-free Sign Language TranslationRepresentation LearningRhythmSign Language Retrieval+1Scaling up Multimodal Pre-training for Sign Language Understanding
Sign language serves as the primary meaning of communication for the deaf-mute community. Different from spoken language, it commonly conveys information by the collaboration of manual features, i.e., hand gestures and b…
Gloss-free Sign Language TranslationSentenceSign Language RecognitionSign Language Retrieval+2Sign Language Translation with Sentence Embedding Supervision
State-of-the-art sign language translation (SLT) systems facilitate the learning process through gloss annotations, either in an end2end manner or by involving an intermediate step. Unfortunately, gloss labelled sign lan…
Gloss-free Sign Language TranslationSentenceSentence EmbeddingSentence-Embedding+3Scaling Sign Language Translation
Sign language translation (SLT) addresses the problem of translating information from a sign language in video to a spoken language in text. Existing studies, while showing progress, are often limited to narrow domains a…
DecoderGloss-free Sign Language TranslationMachine TranslationSign Language Translation+1A Spatio-Temporal Representation Learning as an Alternative to Traditional Glosses in Sign Language Translation and Production
This work addresses the challenges associated with the use of glosses in both Sign Language Translation (SLT) and Sign Language Production (SLP). While glosses have long been used as a bridge between sign language and sp…
Gloss-free Sign Language TranslationRepresentation LearningSelf-Supervised LearningSign Language Production+3Improving Gloss-free Sign Language Translation by Reducing Representation Density
Gloss-free sign language translation (SLT) aims to develop well-performing SLT systems with no requirement for the costly gloss annotations, but currently still lags behind gloss-based approaches significantly. In this p…
Contrastive LearningGloss-free Sign Language TranslationSign Language TranslationTranslationSign2GPT: Leveraging Large Language Models for Gloss-Free Sign Language Translation
Automatic Sign Language Translation requires the integration of both computer vision and natural language processing to effectively bridge the communication gap between sign and spoken languages. However, the deficiency …
Gloss-free Sign Language TranslationSign Language TranslationTranslationLLMs are Good Sign Language Translators
Sign Language Translation (SLT) is a challenging task that aims to translate sign videos into spoken language. Inspired by the strong translation capabilities of large language models (LLMs) that are trained on extensive…
Gloss-free Sign Language TranslationSign Language TranslationTranslationFactorized Learning Assisted with Large Language Model for Gloss-free Sign Language Translation
Previous Sign Language Translation (SLT) methods achieve superior performance by relying on gloss annotations. However, labeling high-quality glosses is a labor-intensive task, which limits the further development of SLT…
Gloss-free Sign Language TranslationLanguage ModelingLanguage ModellingLarge Language Model+2Towards Privacy-Aware Sign Language Translation at Scale
A major impediment to the advancement of sign language translation (SLT) is data scarcity. Much of the sign language data currently available on the web cannot be used for training supervised models due to the lack of al…
Gloss-free Sign Language TranslationSign Language TranslationTranslationGloss-free Sign Language Translation: Improving from Visual-Language Pretraining
Sign Language Translation (SLT) is a challenging task due to its cross-domain nature, involving the translation of visual-gestural language to text. Many previous methods employ an intermediate representation, i.e., glos…
DecoderGloss-free Sign Language TranslationSelf-Supervised LearningSign Language Recognition+2Gloss Attention for Gloss-free Sign Language Translation
Most sign language translation (SLT) methods to date require the use of gloss annotations to provide additional supervision information, however, the acquisition of gloss is not easy. To solve this problem, we first perf…
Gloss-free Sign Language TranslationLanguage ModelingLanguage ModellingSentence+3Gloss-Free End-to-End Sign Language Translation
In this paper, we tackle the problem of sign language translation (SLT) without gloss annotations. Although intermediate representation like gloss has been proven effective, gloss annotations are hard to acquire, especia…
Gloss-free Sign Language TranslationSign Language TranslationTranslation0/1 Deep Neural Networks via Block Coordinate Descent
The step function is one of the simplest and most natural activation functions for deep neural networks (DNNs). As it counts 1 for positive variables and 0 for others, its intrinsic characteristics (e.g., discontinuity a…
10-shot image generation16k2D Object Detection+92Open-Domain Sign Language Translation Learned from Online Video
Existing work on sign language translation - that is, translation from sign language videos into sentences in a written language - has focused mainly on (1) data collected in a controlled environment or (2) data in a spe…
Gloss-free Sign Language TranslationSign Language TranslationTranslation