TY - GEN
T1 - Topic Modeling for Customer Service Chats
AU - Hendry, Darell
AU - Darari, Fariz
AU - Nurfadillah, Raditya
AU - Khanna, Gaurav
AU - Sun, Meng
AU - Condylis, Paul Constantine
AU - Taufik, Natanael
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - Chatbot, as a virtual assistant employed by companies, can provide benefits for its users. Users can communicate directly to the chatbot via a short message, after which the chatbot system spontaneously identifies the intent of the message and responds with relevant actions. Unfortunately, the scope of the chatbot knowledge is limited in handling messages by an increasingly varied group of users. The main consequence of such variation is a change in the composition of the intent labels. This paper focuses on topic modeling in dealing with two tasks: first, to find new intents from user messages that are not yet included in any previous intents; and second, to reorganize existing intents by analyzing the topic model generated. In the analysis, two possible changes in intent compositions are intent merging and splitting. The topic modeling approaches used in this research are LDA as a baseline, as well as state-of-the-art Top2Vec and BERTopic. The labeled datasets for evaluation are taken from one of the major e-commerce companies in Indonesia and four public datasets. We evaluate the topic models by using the metrics of topic coherence, diversity, and quality. Our results show that the BERTopic and Top2Vec topic models produced better evaluation scores than the LDA topic model. Additionally, we perform proportion threshold analysis for reorganizing the intents of the datasets.
AB - Chatbot, as a virtual assistant employed by companies, can provide benefits for its users. Users can communicate directly to the chatbot via a short message, after which the chatbot system spontaneously identifies the intent of the message and responds with relevant actions. Unfortunately, the scope of the chatbot knowledge is limited in handling messages by an increasingly varied group of users. The main consequence of such variation is a change in the composition of the intent labels. This paper focuses on topic modeling in dealing with two tasks: first, to find new intents from user messages that are not yet included in any previous intents; and second, to reorganize existing intents by analyzing the topic model generated. In the analysis, two possible changes in intent compositions are intent merging and splitting. The topic modeling approaches used in this research are LDA as a baseline, as well as state-of-the-art Top2Vec and BERTopic. The labeled datasets for evaluation are taken from one of the major e-commerce companies in Indonesia and four public datasets. We evaluate the topic models by using the metrics of topic coherence, diversity, and quality. Our results show that the BERTopic and Top2Vec topic models produced better evaluation scores than the LDA topic model. Additionally, we perform proportion threshold analysis for reorganizing the intents of the datasets.
KW - chatbot
KW - e-commerce
KW - intent merging
KW - intent reorganization
KW - intent splitting
KW - topic modeling
UR - http://www.scopus.com/inward/record.url?scp=85123865671&partnerID=8YFLogxK
U2 - 10.1109/ICACSIS53237.2021.9631322
DO - 10.1109/ICACSIS53237.2021.9631322
M3 - Conference contribution
AN - SCOPUS:85123865671
T3 - 2021 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2021
BT - 2021 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2021
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 13th International Conference on Advanced Computer Science and Information Systems, ICACSIS 2021
Y2 - 23 October 2021 through 26 October 2021
ER -