Title
Performance comparison of natural language understanding engines in the educational domain
Date Issued
01 January 2020
Access level
open access
Resource Type
journal article
Publisher(s)
Science and Information Organization
Abstract
Recently, chatbots are having a great importance in different domains and are becoming more and more common in customer service. One possible cause is the wide variety of platforms that offer the natural language understanding as a service, for which no programming skills are required. Then, the problem is related to which platform to use to develop a chatbot in the educational domain. Therefore, the main objective of this paper is to compare the main natural language understanding (NLU) engines and determine which could perform better in the educational domain. In this way, researchers can make more justified decisions about which NLU engine to use to develop an educational chatbot. Besides, in this study, six NLU platforms were compared and performance was measured with the F1 score. Training data and input messages were extracted from Mariateguino Bot, which was the chatbot of the Jose Carlos Mariategui University during 2018. The results of this comparison indicates that Watson Assistant has the best performance, with an average F1 score of 0.82, which means that it is able to answer correctly in most cases. Finally, other factors can condition the choice of a natural language understanding engine, so that ultimately the choice is left to the user.
Start page
753
End page
757
Volume
11
Issue
8
Language
English
OCDE Knowledge area
Ciencias de la información
Subjects
Scopus EID
2-s2.0-85091015466
Source
International Journal of Advanced Computer Science and Applications
Resource of which it is part
International Journal of Advanced Computer Science and Applications
ISSN of the container
2158107X
Sources of information:
Directorio de Producción Científica
Scopus