Recognizing Indonesian Sign Language Gestures Using Features Generated by Elliptical Model Tracking and Angular Projection

Drianka Mahdy Adimas, Erdefi Rakun, Dadan Hardianto

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

In this paper, we propose a method of feature extraction applied to the hands, fingers and arms, intended to improve the accuracy of Sistem Isyarat Bahasa Indonesia (SIBI) gesture recognition. Using common smartphone camera, we recorded multiple sequence of gestures used to sign for affixes and root words in SIBI then applied image processing and tracking methods to extract features from the shape and location of the hands. To extract the features, skin color segmentation was applied to separate hands and face blob from the background. Then the object would be registered to an ellipse model and tracked through the videos using elliptical model tracking. Finally, the video was then processed frame by frame, and each successfully tracked object is subjected to angular projection to generate the aforementioned features. The model that was used to recognize SIBI gestures is 2-layers Long Short-Term Memory (LSTM) neural network. Accuracy of the proposed method is measured by comparing the prediction with the actual gesture of the testing data. The highest level of accuracy achieved for the prefix, root and suffix datasets are 91.74%, 98.94%, and 97.71% respectively.

Original languageEnglish
Title of host publicationProceedings - 2019 2nd International Conference on Intelligent Autonomous Systems, ICoIAS 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages25-31
Number of pages7
ISBN (Electronic)9781728126623
DOIs
Publication statusPublished - 1 Feb 2019
Event2nd International Conference on Intelligent Autonomous Systems, ICoIAS 2019 - Singapore, Singapore
Duration: 28 Feb 20192 Mar 2019

Publication series

NameProceedings - 2019 2nd International Conference on Intelligent Autonomous Systems, ICoIAS 2019

Conference

Conference2nd International Conference on Intelligent Autonomous Systems, ICoIAS 2019
Country/TerritorySingapore
CitySingapore
Period28/02/192/03/19

Keywords

  • Computer Vision
  • Deep Learning
  • Feature Extraction
  • Image Processing
  • Machine Learning
  • Pattern Recognition
  • Sign Language

Fingerprint

Dive into the research topics of 'Recognizing Indonesian Sign Language Gestures Using Features Generated by Elliptical Model Tracking and Angular Projection'. Together they form a unique fingerprint.

Cite this