AI-assisted automated scoring of picture-cued writing tasks for language assessment

Ruibin ZHAO, Yipeng ZHUANG, Di ZOU, Qin XIE, Philip L. H. YU*

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

2 Citations (Scopus)

Abstract

Grading assignments is inherently subjective and time-consuming; automatic scoring tools can greatly reduce teacher workload and shorten the time needed for providing feedback to learners. The purpose of this paper is to propose a novel method for automatically scoring student responses to picture-cued writing tasks. As a popular paradigm for language instruction and assessment, a picture-cued writing task typically requires students to describe a picture or pictures. Correspondingly, the automatic scoring methods must measure the link(s) between visual pictures and their textual descriptions. For this purpose, we first designed a picture-cued writing test and collected nearly 4 k responses from 279 K12 students. Based on these responses, we then developed an AI scoring model by incorporating the emerging cross-modal matching technology and some NLP algorithms. The performance of the model was evaluated carefully with six popular measures and was found to demonstrate accurate scoring results with a small mean absolute error of 0.479 and a high adjacent-agreement rate of 90.64%. We believe this method could reduce the subjective elements inherent in human grading and save teachers’ time from the mundane task of grading to other valuable endeavors such as designing teaching plans based on AI-generated diagnosis of student progress.

Original languageEnglish
Pages (from-to)7031-7063
Number of pages33
JournalEducation and Information Technologies
Volume28
Issue number6
DOIs
Publication statusPublished - Jun 2023
Externally publishedYes

Bibliographical note

Funding Information:
This work was supported by the One-off Special Fund from Central and Faculty [grant number 02136] and the Start-Up Research Grant [grant number RG41/20-21R] of the Education University of Hong Kong; and the Youth Elite Supporting Plan in Universities of Anhui Province [grant number gxyqZD2019077], the Higher Education Teaching and Research Project of Anhui Province [grant number 2020jyxm0633], and the Science and Technology Plan Project in Chuzhou [grant number 2021ZD016].

Publisher Copyright:
© 2022, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.

Keywords

  • Artificial intelligence
  • Automated writing assessment
  • Cross-modal matching
  • Picture-cued writing

Fingerprint

Dive into the research topics of 'AI-assisted automated scoring of picture-cued writing tasks for language assessment'. Together they form a unique fingerprint.

Cite this