Continuous Sign Language Recognition System using Deep Learning with MediaPipe Holistic

2024-11-07 · Sharvani Srivastava, Sudhakar Singh, Pooja, Shiv Prakash

Sign languages are the language of hearing-impaired people who use visuals like the hand, facial, and body movements for communication. There are different signs and gestures representing alphabets, words, and phrases. Nowadays approximately 300 sign languages are being practiced worldwide such as American Sign Language (ASL), Chinese Sign Language (CSL), Indian Sign Language (ISL), and many more. Sign languages are dependent on the vocal language of a place. Unlike vocal or spoken languages, there are no helping words in sign language like is, am, are, was, were, will, be, etc. As only a limited population is well-versed in sign language, this lack of familiarity of sign language hinders hearing-impaired people from communicating freely and easily with everyone. This issue can be addressed by a sign language recognition (SLR) system which has the capability to translate the sign language into vocal language. In this paper, a continuous SLR system is proposed using a deep learning model employing Long Short-Term Memory (LSTM), trained and tested on an ISL primary dataset. This dataset is created using MediaPipe Holistic pipeline for tracking face, hand, and body movements and collecting landmarks. The system recognizes the signs and gestures in real-time with 88.23% accuracy.

📄 PDF Abstract BibTeX arXiv:2411.04517

Code (0)

등록된 구현이 없습니다.

Tasks

Sign Language Recognition

Methods 이 논문이 사용한 방법론

American 설명 없음

SLR Please enter a description about the method here

Mediapipe and CNNs for Real-Time ASL Gesture Recognition

2023-05-09 · Rupesh Kumar, Ashutosh Bajpai, Ayush Sinha

This research paper describes a realtime system for identifying American Sign Language (ASL) movements that employs modern computer vision and machine learning approaches. The suggested method makes use of the Mediapipe …

Gesture Recognition Sign Language Recognition

Enhancing Sign Language Detection through Mediapipe and Convolutional Neural Networks (CNN)

2024-06-06 · Aditya Raj Verma, Gagandeep Singh, Karnim Meghwal, Banawath Ramji 외

This research combines MediaPipe and CNNs for the efficient and accurate interpretation of ASL dataset for the real-time detection of sign language. The system presented here captures and processes hands' gestures in rea…

Gesture Recognition

Optimizing Hand Region Detection in MediaPipe Holistic Full-Body Pose Estimation to Improve Accuracy and Avoid Downstream Errors

2024-05-06 · Amit Moryossef

This paper addresses a critical flaw in MediaPipe Holistic's hand Region of Interest (ROI) prediction, which struggles with non-ideal hand orientations, affecting sign language recognition accuracy. We propose a data-dri…

Pose Estimation Sign Language Recognition

Sign Language Recognition using Bidirectional Reservoir Computing

2025-11-30 · Nitin Kumar Singh, Arie Rachmad Syulistyo, Yuichiro Tanaka, Hakaru Tamukoh arxiv

Sign language recognition (SLR) facilitates communication between deaf and hearing individuals. Deep learning is widely used to develop SLR-based systems; however, it is computationally intensive and requires substantial…