Perceived MOOC satisfaction: a review mining approach using machine learning and fine-tuned BERTs

Xieling CHEN, Haoran XIE*, Di ZOU, Gary CHENG, Xiaohui TAO, Fu Lee WANG

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

Abstract

This study investigates the application of machine learning and BERT models to identify topic categories in helpful online course reviews and uncover factors that influence the overall satisfaction of learners in massive open online courses (MOOCs). The research has three main objectives: (1) to assess the effectiveness of machine learning models in classifying review helpfulness, (2) to evaluate the performance of fine-tuned BERT models in identifying review topics, and (3) to explore the factors that influence learner satisfaction across various disciplines. The study uses a MOOC corpus containing 102,184 course reviews from 401 courses across 13 disciplines. The methodology involves three approaches: (1) machine learning for automatic classification of review helpfulness, (2) BERT models for automatic classification of review topics, and (3) multiple linear regression analysis to explore the factors influencing learner satisfaction. The results show that most machine learning models achieve precision, recall, and F1 scores above 80%, 99%, and 89%, respectively, in identifying review helpfulness. The fine-tuned BERT model outperforms baseline models with precision, recall, and F1 scores of 78.4%, 74.4%, and 75.9%, respectively, in classifying review topics. Additionally, the regression analysis identifies key factors affecting learner satisfaction, such as the positive influence of “Instructor” frequency and the negative impact of “Platforms and tools” and “Process”. These insights offer valuable guidance for educators, course designers, and platform developers, contributing to the optimization of MOOC offerings to better meet the evolving needs of learners.
Original languageEnglish
Article number100366
Number of pages17
JournalComputers and Education: Artificial Intelligence
Volume8
Early online date17 Jan 2025
DOIs
Publication statusE-pub ahead of print - 17 Jan 2025

Bibliographical note

This article is partly based on the first author Xieling Chen’s unpublished PhD dissertation entitled “Exploring factors affecting learner satisfaction toward MOOCs: A hybrid approach of topic modeling, deep learning, and structural equation modeling” submitted to The Education University of Hong Kong in 2022.

Publisher Copyright: © 2025

Funding

This work was supported by the National Natural Science Foundation of China (No. 62307010) and the Philosophy and Social Science Planning Project of Guangdong Province of China (No. GD24XJY17), as well as Lam Woo Research Fund (LWP20019) and the Faculty Research Grants (DB23B2 and DB24A4) of Lingnan University, Hong Kong.

Keywords

  • BERT models
  • Learner satisfaction
  • MOOCs
  • Machine learning
  • Multiple linear regression

Fingerprint

Dive into the research topics of 'Perceived MOOC satisfaction: a review mining approach using machine learning and fine-tuned BERTs'. Together they form a unique fingerprint.

Cite this