Assessing Writing Oct 2025

GenAI and human assessments of L2 Chinese writing: Interrater reliability and rater bias

Yuan Lu ; Xiaoying Liles ; Xi Ma
Journal
Assessing Writing
Published
2025-10-01
DOI
10.1016/j.asw.2025.100989
CompPile
Search in CompPile ↗

Citation Context

Cited by in this index (0)

No articles in this index cite this work.

Cites in this index (3)

  1. Wilson et al. (2024)
    Validity of automated essay scores for elementary-age English language learners…
    Assessing Writing
  2. Barrot (2023)
    Using ChatGPT for second language writing: Pitfalls and potentials
    Assessing Writing
  3. Klobucar et al. (2013)
    Automated scoring in context: Rapid assessment for placed students
    Assessing Writing
Also cites 33 works outside this index ↓
  1. Bouziane (2024)
    AI versus human effectiveness in essay evaluation
    Discover Education  ↗
  2. Bridgeman (2012)
    Comparison of human and machine scoring of essays: Differences by gender, ethnicity, and …
    Applied Measurement in Education  ↗
  3. Bui, N.M., & Barrot, J.S.(2025). ChatGPT as an automated essay scoring tool in the writing classrooms: how it…
     ↗
  4. Chen (2008)
    Beyond the design of automated writing evaluation: Pedagogical practices and perceived le…
    Language Learning Technology  ↗
  5. Eckes (2005)
    Examining rater effects in TestDaF writing and speaking performance assessments: A many-f…
    Language Assessment Quarterly  ↗
  6. Eckes (2012)
    Operational rater types in writing assessment: Linking rater cognition to rater behavior
    Language Assessment Quarterly  ↗
  7. Geçkin (2023)
    Assessing second-language academic writing: AI vs. human raters
    Journal of Educational Technology and Online Learning  ↗
  8. Hannah (2023)
    Validity arguments for automated essay scoring of young students’ writing traits
    Language Assessment Quarterly  ↗
  9. Karatay (2024)
    Automated writing evaluation use in second language classrooms: A research synthesis
    System  ↗
  10. Li (2004)
    Topic chains in Chinese discourse
    Discourse Processes  ↗
  11. Li (2021)
    Parataxis or hypotaxis? Choices of taxis in Chinese–English translation
    Lingua  ↗
  12. Li (2014)
    The role of automated writing evaluation holistic scores in the ESL classroom
    System  ↗
  13. Lim (2011)
    The development and maintenance of rating quality in performance writing assessment: A lo…
    Language Testing  ↗
  14. Liu (2016)
    Investigating the application of automated writing evaluation to Chinese undergraduate En…
    CALICO Journal  ↗
  15. Mizumoto (2023)
    Exploring the potential of using an AI language model for automated essay scoring
    Research Methods in Applied Linguistics  ↗
  16. Mizumoto (2024)
    Testing the viability of ChatGPT as a companion in L2 writing accuracy assessment
    Research Methods in Applied Linguistics  ↗
  17. Pfau (2023)
    Exploring the potential of ChatGPT in assessing L2 writing accuracy for research purposes
    Research Methods in Applied Linguistics  ↗
  18. Plonsky (2014)
    How big is “big”? Interpreting effect sizes in L2 research
    Language Learning  ↗
  19. Qian (2020)
    Evaluating China’s Automated Essay Scoring System iWrite
    Journal of Educational Computing Research  ↗
  20. Ramesh (2022)
    An automated essay scoring systems: A systematic literature review
    Artificial Intelligence Review  ↗
  21. Ramineni (2018)
    Understanding mean score differences between the e-rater® automated scoring engine and hu…
    ETS Research Report Series  ↗
  22. Richardson (2021)
    Rise of the machines? The evolving role of Artificial Intelligence (AI) technologies in h…
    London Review of Education  ↗
  23. Schaefer (2008)
    Rater bias patterns in an EFL writing assessment
    Language Testing  ↗
  24. Shi (2023)
    A systematic review of automated writing evaluation systems
    Education and Information Technologies  ↗
  25. Shin (2024)
    Exploratory study on the potential of ChatGPT as a rater of second language writing
    Education and Information Technologies  ↗
  26. Shin (2020)
    More efficient processes for creating automated essay scoring frameworks: A demonstration…
    Language Testing  ↗
  27. Tseng (2023)
    AI-writing tools in education: If you can’t beat them, join them
    Journal of China Computer-Assisted Language Learning  ↗
  28. Uyar (2025)
    Artificial intelligence as an automated essay scoring tool: A focus on ChatGPT
    International Journal of Assessment Tools in Education  ↗
  29. Winke (2013)
    Raters’ L2 background as a potential source of bias in rating oral performance
    Language Testing  ↗
  30. Yamashita (2024)
    An application of many-facet Rasch measurement to evaluate automated essay scoring: A cas…
    Research Methods in Applied Linguistics  ↗
  31. Yamashita (2025)
    Exploring potential biases in GPT-4o’s ratings of English language learners’ essays
    Language Testing  ↗
  32. Zhou (2023)
    Chinese intermediate English learners outdid ChatGPT in deep cohesion: Evidence from Engl…
    System  ↗
  33. Zhu (2025)
    A systematic review of artificial intelligence in language education: Current status and …
    Language Learning Technology  ↗
CrossRef global citation count: 0 View in citation network →