Position and Rotation Invariant Sign Language Recognition from 3D Kinect Data with Recurrent Neural Networks
Sign language is a gesture-based symbolic communication medium among speech and hearing impaired people. It also serves as a communication bridge between non-impaired and impaired populations. Unfortunately, in most situations, a non-impaired person is not well conversant in such symbolic languages restricting the natural information flow between these two categories. Therefore, an automated translation mechanism that seamlessly translates sign language into natural language can be highly advantageous. In this paper, we attempt to perform recognition of 30 basic Indian sign gestures. Gestures are represented as temporal sequences of 3D maps (RGB + depth), each consisting of 3D coordinates of 20 body joints captured by the Kinect sensor. A recurrent neural network (RNN) is employed as the classifier. To improve the classifier's performance, we use geometric transformation for the alignment correction of depth frames. In our experiments, the model achieves 84.81% accuracy.
Code (1)
Tasks
Sign Language RecognitionSimilar Papers 제목 키워드 기반
Rethinking Rotation-Invariant Recognition of Fine-grained Shapes from the Perspective of Contour Points
Rotation-invariant recognition of shapes is a common challenge in computer vision. Recent approaches have significantly improved the accuracy of rotation-invariant recognition by encoding the rotational invariance of sha…
RPR-Net: A Point Cloud-based Rotation-aware Large Scale Place Recognition Network
Point cloud-based large scale place recognition is an important but challenging task for many applications such as Simultaneous Localization and Mapping (SLAM). Taking the task as a point cloud retrieval problem, previou…
Autonomous DrivingPoint Cloud RetrievalRetrievalSimultaneous Localization and MappingDeep Positional and Relational Feature Learning for Rotation-Invariant Point Cloud Analysis
In this paper we propose a rotation-invariant deep network for point clouds analysis. Point-based deep networks are commonly designed to recognize roughly aligned 3D shapes based on point coordinates, but suffer from per…
RRV: A Spatiotemporal Descriptor for Rigid Body Motion Recognition
Motion behaviors of a rigid body can be characterized by a 6-dimensional motion trajectory, which contains position vectors of a reference point on the rigid body and rotations of this rigid body over time. This paper de…
DescriptivePositionTowards Few-shot Entity Recognition in Document Images: A Graph Neural Network Approach Robust to Image Manipulation
Recent advances of incorporating layout information, typically bounding box coordinates, into pre-trained language models have achieved significant performance in entity recognition from document images. Using coordinate…
Graph Neural NetworkImage ManipulationLanguage ModelingLanguage Modelling+1