Recognizing the Components of Inflectional Word Gestures in Indonesian Sign System known as SIBI (Sistem Isyarat Bahasa Indonesia) by using Lip Motion

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Citations (Scopus)

Abstract

Sign language is a method of communication used by people who suffer from hearing impairment. Several countries have invented their own sign language including Indonesia's Sistem Isyarat Bahasa Indonesia (SIBI). The uniqueness of SIBI is that it applies the Indonesian language grammatical structure to the hand motions used and uses facial expressions as the replacement of intonation and lip motions to pronounce a word. From previous studies, to increase the accuracy of sign language recognition it is necessary to supplement the hand tracking with lip reading. This study proposes a model to recognize inflectional word gestures through lip motions. From the proposed model, this research found that the sequence of lip motions are very similar to each other, which resulted in very low accuracy in determining prefix (21,64%), root words (23,87%), and suffixes (22,61%), in SIBI just by analyzing the sequence of lip motions. Using similarity-groupings determined in this study, the accuracy of the model proposed can be increased significantly with the accuracy increasing to prefix (73,94%), root words (83,98%), and suffixes (82,10%). Additionally, another cause for the low accuracy of the model is that the labeling was done according to the hand gestures, this resulted in the frame by frame sequence that was captured to not be representative of the entire lip motions.

Original languageEnglish
Title of host publicationProceeding of 2019 International Conference on Electrical Engineering and Informatics, ICEEI 2019
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages384-389
Number of pages6
ISBN (Electronic)9781728124186
DOIs
Publication statusPublished - Jul 2019
Event7th International Conference on Electrical Engineering and Informatics, ICEEI 2019 - Bandung, Indonesia
Duration: 9 Jul 201910 Jul 2019

Publication series

NameProceedings of the International Conference on Electrical Engineering and Informatics
Volume2019-July
ISSN (Print)2155-6830

Conference

Conference7th International Conference on Electrical Engineering and Informatics, ICEEI 2019
CountryIndonesia
CityBandung
Period9/07/1910/07/19

Keywords

  • Inflectional Words
  • Lip Motion
  • SIBI
  • Sign Language
  • Similarity-Groupings

Fingerprint Dive into the research topics of 'Recognizing the Components of Inflectional Word Gestures in Indonesian Sign System known as SIBI (Sistem Isyarat Bahasa Indonesia) by using Lip Motion'. Together they form a unique fingerprint.

Cite this