The Sign System for Indonesian Language or Sistem Isyarat Bahasa Indonesia? (SIBI) is a sign language system that is used to represent Indonesian language. The referred sign language is a systematic movement of fingers and hands to represent a vocabulary. This paper utilizes the Threshold Conditional Random Field (TCRF) model to identify gesture and nongesture automatically. The generated model is an early model to establish a SIBI translation system automatically. Data that were utilized in this research are Skeleton, Image, and Skeleton Image Combination. Data were processed by implementing TCRF algorithm to provide gesture and non-gesture labels automatically. Several experiments had pointed to the highest accuracy up to 81,5% by using skeletal data as an input in TCRF.