Supporting Mouthing in Signed Languages: New innovations and a proposal for future corpus building
A recurring concern, oft repeated, regarding the quality of signing avatars is the lack of proper facial movements, particularly in actions that involve mouthing. An analysis uncovered three challenges contributing to the problem. The first is a difficulty in devising an algorithmic strategy for generating mouthing due to the rich variety of mouthings in sign language. For example, part or all of a spoken word may be mouthed depending on the sign language, the syllabic structure of the mouthed word, as well as the register of address and discourse setting. The second challenge was technological. Previous efforts to create avatar mouthing have failed to model the timing present in mouthing or have failed to properly model the mouth’s appearance. The third challenge is one of usability. Previous editing systems, when they existed, were time-consuming to use. This paper describes efforts to improve avatar mouthing by addressing these challenges, resulting in a new approach for mouthing animation. The paper concludes by proposing an experiment in corpus building using the new approach.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Transfer Learning from Visual Speech Recognition to Mouthing Recognition in German Sign Language
Sign Language Recognition (SLR) systems primarily focus on manual gestures, but non-manual features such as mouth movements, specifically mouthing, provide valuable linguistic information. This work directly classifies m…
Multi-Task LearningSign Language Recognitionspeech-recognitionSpeech Recognition+2SignMouth: Leveraging Mouthing Cues for Sign Language Translation by Multimodal Contrastive Fusion
Sign language translation (SLT) aims to translate natural language from sign language videos, serving as a vital bridge for inclusive communication. While recent advances leverage powerful visual backbones and large lang…
Sign Language TranslationContrastive LearningImproved Facial Realism through an Enhanced Representation of Anatomical Behavior in Sign Language Avatars
Facial movements and expressions are critical features of signed languages, yet are some of the most challenging to reproduce on signing avatars. Due to the relative lack of research efforts in this area, the facial capa…
Mouthing Recognition with OpenPose in Sign Language
Many avatars focus on the hands and how they express sign language. However, sign language also uses mouth and face gestures to modify verbs, adjectives, or adverbs; these are known as non-manual components of the sign. …
Enhancing Portuguese Sign Language Animation with Dynamic Timing and Mouthing
Current signing avatars are often described as unnatural as they cannot accurately reproduce all the subtleties of synchronized body behaviors of a human signer. In this paper, we propose a new dynamic approach for trans…