An Empirical Comparison of BERT, RoBERTa, and Electra for Fact Verification

Muchammad Naseer, Muhamad Asvial, Riri Fitri Sari

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

We reviewed some features of a number of fact verification techniques by comparing 3 (three) algorithms, namely BERT, RoBERTa, and Electra. These 3 (three) algorithms have different advantages, i.e., BERT and RoBERTa predict hidden words using a huge dataset, and Electra verifies facts by detecting tokens that are replaced in a text or sentence. It is necessary to find the model with a good performance evaluation value to produce the best fact verification results. The evaluation of the performance model in this study uses the F1-Score. Our experimental results show that RoBERTa achieves the best accuracy and F1-Score with a value of 95.4% and 95.3% with the parameter value of epoch of 5 (five) and a batch size of 32. The second position is occupied by BERT, with the best result of accuracy and F1-Score at the same value of 94.3% with the epoch of10 (ten) and a batch size of32. Although it provides a shorter elapsed time, unfortunately, Electra does not outperform other models in fact verification.

Original languageEnglish
Title of host publication3rd International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2021
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages241-246
Number of pages6
ISBN (Electronic)9781728176383
DOIs
Publication statusPublished - 13 Apr 2021
Event3rd International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2021 - Jeju Island, Korea, Republic of
Duration: 13 Apr 202116 Apr 2021

Publication series

Name3rd International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2021

Conference

Conference3rd International Conference on Artificial Intelligence in Information and Communication, ICAIIC 2021
Country/TerritoryKorea, Republic of
CityJeju Island
Period13/04/2116/04/21

Keywords

  • BERT
  • Electra
  • Fact Verification
  • FEVER Dataset
  • RoBERTa

Fingerprint

Dive into the research topics of 'An Empirical Comparison of BERT, RoBERTa, and Electra for Fact Verification'. Together they form a unique fingerprint.

Cite this