TY - GEN
T1 - Recognizing fingerspelling in sibi (sistem Isyarat Bahasa Indonesia) using openpose and elliptical Fourier descriptor
AU - Firdaus, Nanda Maulina
AU - Rakun, Erdefi
N1 - Funding Information:
This work is supported by University of Indonesia’s Research Grant No. NKB-0511/UN2.R3.1/HKP.05.00/2019. This support is gratefully received and acknowledge.
Publisher Copyright:
© 2019 Association for Computing Machinery.
PY - 2019/11/15
Y1 - 2019/11/15
N2 - Humans are social beings; they communicate with other humans to coexist. In general, humans use verbal methods to communicate. However, the limited of verbal communication in people with hearing loss causes them to use non-verbal communication through sign language. It is challenging for nondisabled to grasp sign language quickly because it takes a long time to learn and understand. Therefore, a system that can recognize SIBI is needed. This research focused on fingerspelling gesture in SIBI, concentrating on finger and hand movements. The accuracy results by using OpenPose as feature extraction achieved a 61.49% accuracy. To improve accuracy, by using OpenPose as hand tracking, image masking as pre-processing, and Elliptical Fourier Descriptor as feature extraction, this method achieved a 64.05% accuracy. We investigated whether feature extraction with EFD and OpenPose for hand tracking can improve accuracy; however, some labels still could not be detected correctly. To improve accuracy, image masking and delete similarity frames are used as pre-processing; OpenPose uses as hand tracking, and EFD uses as feature extraction. This method achieved 67.23% accuracy.
AB - Humans are social beings; they communicate with other humans to coexist. In general, humans use verbal methods to communicate. However, the limited of verbal communication in people with hearing loss causes them to use non-verbal communication through sign language. It is challenging for nondisabled to grasp sign language quickly because it takes a long time to learn and understand. Therefore, a system that can recognize SIBI is needed. This research focused on fingerspelling gesture in SIBI, concentrating on finger and hand movements. The accuracy results by using OpenPose as feature extraction achieved a 61.49% accuracy. To improve accuracy, by using OpenPose as hand tracking, image masking as pre-processing, and Elliptical Fourier Descriptor as feature extraction, this method achieved a 64.05% accuracy. We investigated whether feature extraction with EFD and OpenPose for hand tracking can improve accuracy; however, some labels still could not be detected correctly. To improve accuracy, image masking and delete similarity frames are used as pre-processing; OpenPose uses as hand tracking, and EFD uses as feature extraction. This method achieved 67.23% accuracy.
KW - Elliptical Fourier Descriptor
KW - Fingerspelling
KW - Non-verbal communication
KW - OpenPose
KW - SIBI
UR - http://www.scopus.com/inward/record.url?scp=85123043063&partnerID=8YFLogxK
U2 - 10.1145/3373477.3373491
DO - 10.1145/3373477.3373491
M3 - Conference contribution
AN - SCOPUS:85123043063
T3 - ACM International Conference Proceeding Series
BT - Proceedings of the International Conference on Advanced Information Science and System, AISS 2019
PB - Association for Computing Machinery
T2 - 2019 International Conference on Advanced Information Science and System, AISS 2019
Y2 - 15 November 2019 through 17 November 2019
ER -