Continual Learning of Deep Learning for Indonesian Sentiment Analysis

Carlo Johan Nikanor, Hendri Murfi, Muhammad Adani Osmardifa, Gianinna Ardaneswari

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

Abstract

High-level social media usage makes this social media frequently used as one of the sources for sentiment analysis. Sentiment analysis is a field of study that analyzes people’s opinions or evaluations of entities such as products and services. The Bidirectional Encoder Representation from Transformers (BERT) model is a deep learning architecture that achieves state-of-the-art performance for many natural language processing problems, including sentiment analysis. Several further developments have implemented continual learning on the deep learning model. By applying continual learning, the deep learning model continuously learns based on new data while retaining previously learned knowledge. In this paper, we analyze the performance of the BERT model for continual learning in some domains of Indonesian sentiment analysis. Then it will be compared with two standard deep learning models: fine-tuned embedding with CNN and fine-tuned embedding with LSTM. Our simulation shows the BERT model gives the best accuracy for the transfer of knowledge. However, the fine-tuned embedding with LSTM model is better for retain of knowledge. Moreover, our simulation shows that the order of the source domains affects the performance of BERT for both transfer of knowledge and retain of knowledge.

Original languageEnglish
Title of host publicationSignals and Communication Technology
PublisherSpringer Science and Business Media Deutschland GmbH
Pages13-26
Number of pages14
DOIs
Publication statusPublished - 2024

Publication series

NameSignals and Communication Technology
VolumePart F1703
ISSN (Print)1860-4862
ISSN (Electronic)1860-4870

Keywords

  • BERT
  • Continual learning
  • Deep learning
  • Retain of knowledge
  • Sentiment analysis
  • Transfer of knowledge

Fingerprint

Dive into the research topics of 'Continual Learning of Deep Learning for Indonesian Sentiment Analysis'. Together they form a unique fingerprint.

Cite this