Individualized feedback to raters in language assessment : Impacts on rater effects

Jing HUANG*, Gaowei CHEN

*Corresponding author for this work

Research output: Journal PublicationsJournal Article (refereed)peer-review

3 Citations (Scopus)


This study examined the impacts of individualized feedback on rater effects in performance-based language assessment. The raters were 93 native Chinese speakers without previous rating experience, and they were randomly assigned to one of the three treatment groups. The three groups of raters differed in the way that they received individualized feedback in a given period: (a) a control group receiving no feedback, (b) a single-feedback group receiving the feedback once, and (c) a double-feedback group receiving the feedback twice. The results indicated that individualized feedback significantly reduced rater severity/leniency and rater misfit. Specifically, raters from both the double-feedback and single-feedback groups exhibited lower levels of rater severity/leniency, compared with those from the control group. Raters from the double-feedback group showed lower levels of rater misfit than those from the single-feedback group. With regard to the retention of reduction in rater effects, individualized feedback was found to assist raters from the double-feedback group to maintain the reduction in rater severity/leniency. It also helped raters from the single-feedback group to maintain the reduction in rater misfit. These findings may shed light on the applications of individualized feedback in the designs of face-to-face and online rater training programs.

Original languageEnglish
Article number100623
JournalAssessing Writing
Publication statusPublished - Apr 2022
Externally publishedYes

Bibliographical note

Funding Information:
The study reported in this article was supported by the TOEFL Small Grants for Doctoral Research in Second or Foreign Language Assessment from Educational Testing Service and Hong Kong Research Grants Council, University Grants Committee (Grant No. 17608318 ). I would like to thank the editors and anonymous reviewers for their valuable comments on an earlier draft of this article. My thanks also go to the participants who contributed to the study.

Publisher Copyright:
© 2022 Elsevier Inc.


  • Individualized feedback
  • Many-facet Rasch measurement
  • Rater bias
  • Rater misfit
  • Rater severity/leniency
  • Writing assessment


Dive into the research topics of 'Individualized feedback to raters in language assessment : Impacts on rater effects'. Together they form a unique fingerprint.

Cite this