Combining depth image and skeleton data from Kinect for recognizing words in the sign system for Indonesian language (SIBI [Sistem Isyarat Bahasa Indonesia])

Erdefi Rakun, Mirna Andriani, I. Wayan Wiprayoga, Ken Danniswara, Andros Tjandra

Research output: Contribution to conferencePaperpeer-review

30 Citations (Scopus)

Abstract

The Sign System for Indonesian Language (SIBI) is a rather complex sign language. It has four components that distinguish the meaning of the sign language and it follows the syntax and the grammar of the Indonesian language. This paper proposes a model for recognizing the SIBI words by using Microsoft Kinect as the input sensor. This model is a part of automatic translation from SIBI to text. The features for each word are extracted from skeleton and color-depth data produced by Kinect. Skeleton data features indicate the angle between human joints and Cartesian axes. Color images are transformed to gray-scale and their features are extracted by using Discrete Cosine Transform (DCT) with Cross Correlation (CC) operation. The image's depth features are extracted by running MATLAB regionprops function to get its region properties. The Generalized Learning Vector Quantization (GLVQ) and Random Forest (RF) training algorithm from WEKA data mining tools are used as the classifier of the model. Several experiments with different scenarios have shown that the highest accuracy (96,67%) is obtained by using 30 frames for skeleton combined with 20 frames for region properties image classified by Random Forest.

Original languageEnglish
Pages387-392
Number of pages6
DOIs
Publication statusPublished - 1 Jan 2013
Event2013 5th International Conference on Advanced Computer Science and Information Systems, ICACSIS 2013 - Bali, Indonesia
Duration: 28 Sep 201329 Sep 2013

Conference

Conference2013 5th International Conference on Advanced Computer Science and Information Systems, ICACSIS 2013
CountryIndonesia
CityBali
Period28/09/1329/09/13

Fingerprint Dive into the research topics of 'Combining depth image and skeleton data from Kinect for recognizing words in the sign system for Indonesian language (SIBI [Sistem Isyarat Bahasa Indonesia])'. Together they form a unique fingerprint.

Cite this