Recognition of Sign Language System for Indonesian Language Using Long Short-Term Memory Neural Networks

SIBI (Sign Language System for the Indonesian Language) is the official sign language system for the Indonesian language. This research aims to find a suitable model for performing SIBI-to-text translation on inflectional word gestures. Extant research has been able to translate the alphabet, root words, and numbers from SIBI to text. Inflectional words are root words with prefixes, infixes, and suffixes, or some combination of the three. A new method that splits an inflectional word into three feature vector sets was developed. This reduces the amount of feature sets used, which would otherwise be as big as the product of the prefixes, suffixes, and root words feature sets of the inflectional word gestures. Long Short-Term Memory (LSTM) is used, as this model can take entire sequences as input and does not have to rely on pre-clustered per-frame data. LSTM suits this system well as the SIBI sequence data has a long-term temporal dependency. The 2-layer LSTM performed the best, being 95.4% accurate with root words. The same model is 77% accurate with inflectional words, using the combined skeleton-image feature set, with an 800-epoch limit. The lower accuracy with inflectional words is due to difficulties in recognizing prefixes and suffixes. Keywords: Inflectional Words, Long Short-Term Memory, Deep Learning, Kinect, sign language, SIBI.