Automated writing evaluation systems: A systematic review of Grammarly, Pigai, and Criterion with a perspective on future directions in the age of generative artificial intelligence

With the burgeoning popularity and swift advancements of automated writing evaluation (AWE) systems in language classrooms, scholarly and practical interest in this area has noticeably increased. This systematic review aims to comprehensively investigate current research on three prominent AWE systems: Grammarly, Pigai, and Criterion. Objectives include assessing each system’s characteristics, advantages, and drawbacks, analyzing prior studies’ frameworks, methodologies, findings, and implications, and identifying research gaps and future directions. The analysis of 39 articles underscored an escalating interest in scrutinizing AWE systems, predominantly focusing on their efficacy and learners’ viewpoints. The findings demonstrated the positive impact of AWE systems on enhancing students’ writing proficiency, with both learners and educators conveying positive attitudes towards these digital tools. However, several noteworthy research gaps endure, including the need to further investigate the usage patterns of AWE tools, expanding the participants to wider language proficiency and research comparing AWE feedback with peer feedback. The majority of the studies focused on non-native English-speaking university students over a single academic semester, using quantitative and mixed research methods. The review concludes by offering insights and recommendations for educators and researchers in the field, stressing the importance of tackling the identified research gaps and further delving into the potential of AWE systems in the age of generative artificial intelligence.