Recognizing Indonesian Sign Language Gestures Using Features Generated by Elliptical Model Tracking and Angular Projection

In this paper, we propose a method of feature extraction applied to the hands, fingers and arms, intended to improve the accuracy of Sistem Isyarat Bahasa Indonesia (SIBI) gesture recognition. Using common smartphone camera, we recorded multiple sequence of gestures used to sign for affixes and root words in SIBI then applied image processing and tracking methods to extract features from the shape and location of the hands. To extract the features, skin color segmentation was applied to separate hands and face blob from the background. Then the object would be registered to an ellipse model and tracked through the videos using elliptical model tracking. Finally, the video was then processed frame by frame, and each successfully tracked object is subjected to angular projection to generate the aforementioned features. The model that was used to recognize SIBI gestures is 2-layers Long Short-Term Memory (LSTM) neural network. Accuracy of the proposed method is measured by comparing the prediction with the actual gesture of the testing data. The highest level of accuracy achieved for the prefix, root and suffix datasets are 91.74%, 98.94%, and 97.71% respectively.