Recognizing fingerspelling in sibi (sistem Isyarat Bahasa Indonesia) using openpose and elliptical Fourier descriptor

Nanda Maulina Firdaus, Erdefi Rakun

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Humans are social beings; they communicate with other humans to coexist. In general, humans use verbal methods to communicate. However, the limited of verbal communication in people with hearing loss causes them to use non-verbal communication through sign language. It is challenging for nondisabled to grasp sign language quickly because it takes a long time to learn and understand. Therefore, a system that can recognize SIBI is needed. This research focused on fingerspelling gesture in SIBI, concentrating on finger and hand movements. The accuracy results by using OpenPose as feature extraction achieved a 61.49% accuracy. To improve accuracy, by using OpenPose as hand tracking, image masking as pre-processing, and Elliptical Fourier Descriptor as feature extraction, this method achieved a 64.05% accuracy. We investigated whether feature extraction with EFD and OpenPose for hand tracking can improve accuracy; however, some labels still could not be detected correctly. To improve accuracy, image masking and delete similarity frames are used as pre-processing; OpenPose uses as hand tracking, and EFD uses as feature extraction. This method achieved 67.23% accuracy.

Original languageEnglish
Title of host publicationProceedings of the International Conference on Advanced Information Science and System, AISS 2019
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450372916
DOIs
Publication statusPublished - 15 Nov 2019
Event2019 International Conference on Advanced Information Science and System, AISS 2019 - Singapore, Singapore
Duration: 15 Nov 201917 Nov 2019

Publication series

NameACM International Conference Proceeding Series

Conference

Conference2019 International Conference on Advanced Information Science and System, AISS 2019
Country/TerritorySingapore
CitySingapore
Period15/11/1917/11/19

Keywords

  • Elliptical Fourier Descriptor
  • Fingerspelling
  • Non-verbal communication
  • OpenPose
  • SIBI

Fingerprint

Dive into the research topics of 'Recognizing fingerspelling in sibi (sistem Isyarat Bahasa Indonesia) using openpose and elliptical Fourier descriptor'. Together they form a unique fingerprint.

Cite this