American Sign Language to Text Translation using Transformer and Seq2Seq with LSTM
Sign language translation is one of the important issues in communication between deaf and hearing people, as it expresses words through hand, body, and mouth movements. American Sign Language is one of the sign languages used, one of which is the alphabetic sign. The development of neural machine translation technology is moving towards sign language translation. Transformer became the state-of-the-art in natural language processing. This study compares the Transformer with the Sequence-to-Sequence (Seq2Seq) model in translating sign language to text. In addition, an experiment was conducted by adding Residual Long Short-Term Memory (ResidualLSTM) in the Transformer. The addition of ResidualLSTM to the Transformer reduces the performance of the Transformer model by 23.37% based on the BLEU Score value. In comparison, the Transformer itself increases the BLEU Score value by 28.14 compared to the Seq2Seq model.
Code (0)
등록된 구현이 없습니다.
Tasks
Machine TranslationSign Language TranslationTranslationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Fingerspelling PoseNet: Enhancing Fingerspelling Translation with Pose-Based Transformer Models
We address the task of American Sign Language fingerspelling translation using videos in the wild. We exploit advances in more accurate hand pose estimation and propose a novel architecture that leverages the transformer…
DecoderHand Pose EstimationLanguage ModelingLanguage Modelling+4ADAT: Time-Series-Aware Adaptive Transformer Architecture for Sign Language Translation
Current sign language machine translation systems rely on recognizing hand movements, facial expressions and body postures, and natural language processing, to convert signs into text. Recent approaches use Transformer a…
DecoderMachine TranslationSign Language TranslationTime Series+1Machine Translation between Spoken Languages and Signed Languages Represented in SignWriting
This paper presents work on novel machine translation (MT) systems between spoken and signed languages, where signed languages are represented in SignWriting, a sign language writing system. Our work seeks to address the…
Machine TranslationSign Language TranslationTranslationLarge Sign Language Models: Toward 3D American Sign Language Translation
We present Large Sign Language Models (LSLM), a novel framework for translating 3D American Sign Language (ASL) by leveraging Large Language Models (LLMs) as the backbone, which can benefit hearing-impaired individuals' …
Sign Language TranslationSign Language RecognitionBidirectional American Sign Language to English Translation
We outline a bidirectional translation system that converts sentences from American Sign Language (ASL) to English, and vice versa. To perform machine translation between ASL and English, we utilize a generative approach…
Machine TranslationTranslationWord Alignment