Topic Modeling for Customer Service Chats

Darell Hendry, Fariz Darari, Raditya Nurfadillah, Gaurav Khanna, Meng Sun, Paul Constantine Condylis, Natanael Taufik

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)

Abstract

Chatbot, as a virtual assistant employed by companies, can provide benefits for its users. Users can communicate directly to the chatbot via a short message, after which the chatbot system spontaneously identifies the intent of the message and responds with relevant actions. Unfortunately, the scope of the chatbot knowledge is limited in handling messages by an increasingly varied group of users. The main consequence of such variation is a change in the composition of the intent labels. This paper focuses on topic modeling in dealing with two tasks: first, to find new intents from user messages that are not yet included in any previous intents; and second, to reorganize existing intents by analyzing the topic model generated. In the analysis, two possible changes in intent compositions are intent merging and splitting. The topic modeling approaches used in this research are LDA as a baseline, as well as state-of-the-art Top2Vec and BERTopic. The labeled datasets for evaluation are taken from one of the major e-commerce companies in Indonesia and four public datasets. We evaluate the topic models by using the metrics of topic coherence, diversity, and quality. Our results show that the BERTopic and Top2Vec topic models produced better evaluation scores than the LDA topic model. Additionally, we perform proportion threshold analysis for reorganizing the intents of the datasets.

Original languageEnglish
Title of host publication2021 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665442640
DOIs
Publication statusPublished - 2021
Event13th International Conference on Advanced Computer Science and Information Systems, ICACSIS 2021 - Depok, Indonesia
Duration: 23 Oct 202126 Oct 2021

Publication series

Name2021 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2021

Conference

Conference13th International Conference on Advanced Computer Science and Information Systems, ICACSIS 2021
Country/TerritoryIndonesia
CityDepok
Period23/10/2126/10/21

Keywords

  • chatbot
  • e-commerce
  • intent merging
  • intent reorganization
  • intent splitting
  • topic modeling

Fingerprint

Dive into the research topics of 'Topic Modeling for Customer Service Chats'. Together they form a unique fingerprint.

Cite this