Automatic Gloss-level Data Augmentation for Sign Language Translation
Securing sufficient data to enable automatic sign language translation modeling is challenging. The data insufficiency issue exists in both video and text modalities; however, fewer studies have been performed on text data augmentation compared to video data. In this study, we present three methods of augmenting sign language text modality data, comprising 3,052 Gloss-level Korean Sign Language (GKSL) and Word-level Korean Language (WKL) sentence pairs. Using each of the three methods, the following number of sentence pairs were created: blank replacement 10,654, sentence paraphrasing 1,494, and synonym replacement 899. Translation experiment results using the augmented data showed that when translating from GKSL to WKL and from WKL to GKSL, Bi-Lingual Evaluation Understudy (BLEU) scores improved by 0.204 and 0.170 respectively, compared to when only the original data was used. The three contributions of this study are as follows. First, we demonstrated that three different augmentation techniques used in existing Natural Language Processing (NLP) can be applied to sign language. Second, we propose an automatic data augmentation method which generates quality data by utilizing the Korean sign language gloss dictionary. Lastly, we publish the Gloss-level Korean Sign Language 13k dataset (GKSL13k), which has verified data quality through expert reviews.
Code (0)
등록된 구현이 없습니다.
Tasks
Data AugmentationSentenceSign Language TranslationTranslationSimilar Papers 제목 키워드 기반
Context-gloss Augmentation for Improving Word Sense Disambiguation
The goal of Word Sense Disambiguation (WSD) is to identify the sense of a polysemous word in a specific context. Deep-learning techniques using BERT have achieved very promising results in the field and different methods…
Data AugmentationSentenceTranslationWord Sense DisambiguationIntroducing A Bangla Sentence - Gloss Pair Dataset for Bangla Sign Language Translation and Research
Bangla Sign Language (BdSL) translation represents a low-resource NLP task due to the lack of large-scale datasets that address sentence-level translation. Correspondingly, existing research in this field has been limite…
Sign Language TranslationPrompt EngineeringAutomatic interlinear glossing as two-level sequence classification
Explore More Guidance: A Task-aware Instruction Network for Sign Language Translation Enhanced with Data Augmentation
Sign language recognition and translation first uses a recognition module to generate glosses from sign language videos and then employs a translation module to translate glosses into spoken sentences. Most existing work…
Data AugmentationSign Language RecognitionSign Language TranslationTranslationData Augmentation for Sign Language Gloss Translation
Sign language translation (SLT) is often decomposed into video-to-gloss recognition and gloss-to-text translation, where a gloss is a sequence of transcribed spoken-language words in the order in which they are signed. W…
Data AugmentationLow Resource Neural Machine TranslationLow-Resource Neural Machine TranslationLow Resource NMT+4