Solving Complex Background Problem Using RetinaNet for Sign System for Indonesian Language (SIBI) Gesture-to-Text Translator

Median Hardiv Nugraha, Erdefi Rakun

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)

Abstract

SIBI is the standardized sign language system offi-cially used in Indonesia. The application of SIBI is often found to be a hindrance because there are too many gestures that must be memorized. A mobile-based application is needed as gesture-to-text translator. From Rakun et al., Skin Color Segmentation was used as a method to segment hand and facial features using greenscreen background as dataset (3.367% of WER and 80.180% of SAcc). When this application is used, the gesture video is recorded on complex background but performed poorly (135.180% of WER and 0% of SAcc score). The computational time using Skin Color Segmentation is 0.013 s per frame. OpenPose was used to locate hand and facial position. OpenPose can give better performance (6.312% of WER and 69.293% of SAcc score) compared to Skin Color Segmentation but cannot be implemented on mobile application. The computational time using OpenPose is 0.410 s per frame. The focus of this study is to find a model that can locate hand and facial position on complex background and also can be implemented on mobile application. The model we use is RetinaNet. RetinaNet is proven to locate hand and facial position much better (4,100% of WER and 78,990 % of SAcc score) than Skin Color Segmentation and OpenPose. The computational time using RetinaNet is 0.038 s per frame.

Original languageEnglish
Title of host publicationProceedings - ICACSIS 2022
Subtitle of host publication14th International Conference on Advanced Computer Science and Information Systems
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages45-52
Number of pages8
ISBN (Electronic)9781665489362
DOIs
Publication statusPublished - 2022
Event14th International Conference on Advanced Computer Science and Information Systems, ICACSIS 2022 - Virtual, Online, Indonesia
Duration: 1 Oct 20223 Oct 2022

Publication series

NameProceedings - ICACSIS 2022: 14th International Conference on Advanced Computer Science and Information Systems

Conference

Conference14th International Conference on Advanced Computer Science and Information Systems, ICACSIS 2022
Country/TerritoryIndonesia
CityVirtual, Online
Period1/10/223/10/22

Keywords

  • Computer Vision
  • Deep Learning
  • Image Processing
  • Object Detection
  • RetinaNet
  • SIBI
  • Sign Language

Fingerprint

Dive into the research topics of 'Solving Complex Background Problem Using RetinaNet for Sign System for Indonesian Language (SIBI) Gesture-to-Text Translator'. Together they form a unique fingerprint.

Cite this