ABSA of Indonesian customer reviews using IndoBERT: single-sentence and sentence-pair classification approaches

Aspect-based sentiment analysis (ABSA) task is important to identify user satisfaction from customer reviews by recognizing the sentiments of all aspects discussed in the reviews. This work investigates a novel study on the effectiveness and efficiency of three IndoBERT-based models for solving the ABSA task in Indonesian language. IndoBERT is a state-of-the-art transformer-based model, i.e., bidirectional encoder representations from transformers (BERT), that was pre-trained on Indonesian language. Our first model utilizes IndoBERT in a feature-based mode, paired with the convolutional neural network (CNN) and machine learning models, for single-sentence classification. Next, our second model is obtained by fine-tuning the IndoBERT model for a typical single-sentence classification to build an end-to-end model. At last, our third model also adopts a fine-tuning approach to use IndoBERT, but for sentence-pair classification by utilizing auxiliary sentences. Our results demonstrate that the third model, the fine-tuned IndoBERT for sentence-pair classification, gains the highest effectiveness. It demonstrates significant improvement over deep learning baselines (Word2Vec-CNN-XGBoost) by 23.6% and transformer-based baselines (mBERT-aux-NLIB) by 2.2% in terms of F-1 score. When considering both effectiveness and efficiency, the results show that the best-performing model is our second model, the fine-tuned IndoBERT for single-sentence classification.