Leveraging text mining and analytic hierarchy process for the automatic evaluation of online courses

Xieling CHEN, Haoran XIE, Xiaohui TAO, Fu Lee WANG*, Jie CAO

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

Abstract

This study introduced a multi-criteria decision-making methodology leveraging text mining and analytic hierarchy process (AHP) for online course quality evaluation based on students’ feedback texts. First, a hierarchical structure of online course evaluation criteria was formulated by integrating topics (sub-criteria) identified through topic modeling and interpreted based on transactional distance and technology acceptance theories. Second, the weights of the criteria in the hierarchical structure were determined based on topic proportions. Third, the AHP was employed to determine the overall relative advantage of online courses and their relative advantage within each criterion based on the hierarchical framework and criterion weights. The proposed approach was implemented on the datasets of 6940 reviews for knowledge-seeking courses in Art, Design, and Humanities (D1) and 44,697 reviews for skill-seeking courses in Computer Science, Engineering, and Programming (D2) from Class Central to determine ranking positions of nine courses from both D1 and D2 as alternatives. Results revealed common concerns among knowledge and skill-seeking course learners, encompassing “assessment”, “content”, “effort”, “usefulness”, “enjoyment”, “faculty”, “interaction”, and “structure”. The article provides valuable insights into the online course evaluation and selection processes for learners in D1 and D2 groups. Notably, both groups prioritize “effort” and “faculty”, while D2 learners value “assessment” and “enjoyment”, and D1 learners value “usefulness” more. This study demonstrates the efficacy of leveraging online learner reviews and topic modeling for automating MOOC evaluation and informing learners’ decision-making processes.
Original languageEnglish
JournalInternational Journal of Machine Learning and Cybernetics
Early online date21 May 2024
DOIs
Publication statusE-pub ahead of print - 21 May 2024

Bibliographical note

Publisher Copyright:
© The Author(s) 2024.

Keywords

  • Analytic hierarchy process (AHP)
  • Automatic evaluation
  • Course selection
  • MOOCs
  • Topic mining

Fingerprint

Dive into the research topics of 'Leveraging text mining and analytic hierarchy process for the automatic evaluation of online courses'. Together they form a unique fingerprint.

Cite this