Multi-label text classification of Indonesian customer reviews using bidirectional encoder representations from transformers language model

Nuzulul Khairu Nissa, Evi Yulianti

Research output: Contribution to journalArticlepeer-review

Abstract

Customer review is a critical resource to support the decision-making process in various industries. To understand how customers perceived each aspect of the product, we can first identify all aspects discussed in the customer reviews by performing multi-label text classification. In this work, we want to know the effectiveness of our two proposed strategies using bidirectional encoder representations from transformers (BERT) language model that was pre-trained on the Indonesian language, referred to as IndoBERT, to perform multi-label text classification. First, IndoBERT is used as feature representation to be combined with convolutional neural network-extreme gradient boosting (CNN-XGBoost). Second, IndoBERT is used both as the feature representation as well as the classifier to directly solve the classification task. Additional analysis is performed to compare our results with those using multilingual BERT model. According to our experimental results, our first model using IndoBERT as feature representation shows significant performance over some baselines. Our second model using IndoBERT as both feature representation and classifier can significantly enhance the effectiveness of our first model. In summary, our proposed models can improve the effectiveness of the baseline using Word2Vec-CNN-XGBoost by 19.19% and 6.17%, in terms of accuracy and F-1 score, respectively.

Original languageEnglish
Pages (from-to)5641-5652
Number of pages12
JournalInternational Journal of Electrical and Computer Engineering
Volume13
Issue number5
DOIs
Publication statusPublished - Oct 2023

Keywords

  • Convolutional neural network
  • Customer review
  • IndoBERT
  • Multi-label text classification
  • Word2Vec

Fingerprint

Dive into the research topics of 'Multi-label text classification of Indonesian customer reviews using bidirectional encoder representations from transformers language model'. Together they form a unique fingerprint.

Cite this