Generating of SIBI animated gestures from indonesian text

E. Rakun, I. S.A. Darmana

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Sign System for Bahasa Indonesia (SIBI) is the official sign language authorised by The Ministry of Education and Culture of Indonesia and being used as one of the communication media by School for Children with Special Needs (SLB) for people with hearing impairments in the process of learning. For people who have a lack of knowledge about SIBI gestures certainly will have difficulty to communicate with people with hearing impairments. Thus, a translation system from SIBI gestures to sentences in Bahasa Indonesia or vice versa is needed. This research is the initial stage of a translation system from sentences in Bahasa Indonesia to SIBI Gestures. The focus of this research is to generate sign gestures in the form of 3D Animation from a sentence input in text format and deployed on the smartphone device. The generation process starts from deconstructing the input sentence into its word components using a look-up table that consists of affixes, root words, and a "slang"dictionary. Then, this word components are referred to their corresponding gesture animations. The gesture data are recorded with motion-capture sensor "Perception Neuron v2"and use the official SIBI Dictionary as the reference. The animation is rendered in Unity engine and to create a smooth transition between one gesture animation to another, a LERP (Linear Interpolation) based method is implemented using Animancer API. Based on evaluation results, produced gestures correctly represent smooth SIBI gestures with the most significant accuracy score of 97.56% with a user satisfaction score of Very Satisfied is 84%, Satisfied is 14%, and Fair is 2%.

Original languageEnglish
Title of host publicationICEMT 2020 - 2020 the 4th International Conference on Education and Multimedia Technology
PublisherAssociation for Computing Machinery
Pages256-264
Number of pages9
ISBN (Electronic)9781450388375
DOIs
Publication statusPublished - 19 Jul 2020
Event4th International Conference on Education and Multimedia Technology, ICEMT 2020 - Virtual, Online, Japan
Duration: 19 Jul 202022 Jul 2020

Publication series

NameACM International Conference Proceeding Series

Conference

Conference4th International Conference on Education and Multimedia Technology, ICEMT 2020
Country/TerritoryJapan
CityVirtual, Online
Period19/07/2022/07/20

Fingerprint

Dive into the research topics of 'Generating of SIBI animated gestures from indonesian text'. Together they form a unique fingerprint.

Cite this