Usability Evaluation of the MeMo Tutor: A Scaffolding-Based Pedagogical Agent to Facilitate Learning

Nowadays, distance learning faces challenges in increasing students' cognitive engagement and motivation. Research on pedagogical agents is one of the solutions offered to overcome the challenges. MeMo Tutor, the pedagogical agent in this study, acts as a scaffold using integrated metacognitive and motivational scaffoldings. This study was conducted to evaluate the usability of the MeMo Tutor by measuring the level of user satisfaction when using the application. Measuring usability helps determine how usable (e.g., effective, efficient, easy to learn) the system from users' perspective, determine whether any issues need to be fixed, and monitor the performance of the system design over time. The usability evaluation process involved thirteen participants in the first iteration and fifty-six participants in the second iteration. Evaluation data were collected and analyzed using an online prototype analytical tool and System Usability Scale (SUS). In the first iteration, the usability score shows that users were often confused by ambiguous words, especially if the words were close together. This study shows that the average usability score of the online prototype analytical tool correlates well with the average SUS score in the second iteration. The usability results obtained were 88 fromthe online prototype analytical tool and 81.16 from SUS. The score indicates that users were satisfied with the MeMo Tutor application. We recommend focusing on developing mobile-based or responsive web interfaces to improve user experience. In addition, it is crucial to analyze the relationship between the learner's experience and the learning achievement obtained by the user after using the MeMo Tutor.