Automated writing evaluation systems: A systematic review of Grammarly, Pigai, and Criterion with a perspective on future directions in the age of generative artificial intelligence

Linqian DING, Di ZOU*

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

Abstract

With the burgeoning popularity and swift advancements of automated writing evaluation (AWE) systems in language classrooms, scholarly and practical interest in this area has noticeably increased. This systematic review aims to comprehensively investigate current research on three prominent AWE systems: Grammarly, Pigai, and Criterion. Objectives include assessing each system’s characteristics, advantages, and drawbacks, analyzing prior studies’ frameworks, methodologies, findings, and implications, and identifying research gaps and future directions. The analysis of 39 articles underscored an escalating interest in scrutinizing AWE systems, predominantly focusing on their efficacy and learners’ viewpoints. The findings demonstrated the positive impact of AWE systems on enhancing students’ writing proficiency, with both learners and educators conveying positive attitudes towards these digital tools. However, several noteworthy research gaps endure, including the need to further investigate the usage patterns of AWE tools, expanding the participants to wider language proficiency and research comparing AWE feedback with peer feedback. The majority of the studies focused on non-native English-speaking university students over a single academic semester, using quantitative and mixed research methods. The review concludes by offering insights and recommendations for educators and researchers in the field, stressing the importance of tackling the identified research gaps and further delving into the potential of AWE systems in the age of generative artificial intelligence.
Original languageEnglish
JournalEducation and Information Technologies
DOIs
Publication statusE-pub ahead of print - 3 Jan 2024

Bibliographical note

Publisher Copyright:
© 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

Keywords

  • Artificial intelligence
  • Automated writing evaluation
  • Criterion
  • Feedback
  • Grammarly
  • Pigai

Fingerprint

Dive into the research topics of 'Automated writing evaluation systems: A systematic review of Grammarly, Pigai, and Criterion with a perspective on future directions in the age of generative artificial intelligence'. Together they form a unique fingerprint.

Cite this