Assessing Writing Apr 2026

Evaluating the consistency between human raters and three AI systems on the scoring of argumentative essays

SeongYeup Kim ; Matthew T. McCrudden
Journal
Assessing Writing
Published
2026-04-01
DOI
10.1016/j.asw.2026.101031
CompPile
Search in CompPile ↗

Citation Context

Cited by in this index (0)

No articles in this index cite this work.

Cites in this index (1)

  1. Perelman (2014)
    When “the state of the art” is counting words
    Assessing Writing
Also cites 37 works outside this index ↓
  1. Ahia, O., Kumar, S., Gonen, H., Kasai, J., Mortensen, D.R., Smith, N.A., & Tsvetkov, Y. (2023). Do all langua…
     ↗
  2. Attali (2013)
    Scoring with the computer: Alternative procedures for improving the reliability of holist…
    Language Testing  ↗
  3. Banihashem (2024)
    Feedback sources in essay writing: peer-generated or AI-generated feedback?
    International Journal of Educational Technology in Higher Education  ↗
  4. Bouziane (2024)
    AI versus human effectiveness in essay evaluation
    Discover Education  ↗
  5. Broad, B. (2003). What we really value: Beyond rubrics in Teaching and Assessing Writing All USU Press Public…
     ↗
  6. Bucol (2025)
    Exploring ChatGPT as a writing assessment tool
    Innovations in Education and Teaching International  ↗
  7. Bui (2024)
    ChatGPT as an automated essay scoring tool in the writing classrooms: how it compares wit…
    Education and Information Technologies  ↗
  8. Correnti (2022)
    Building a validity argument for an automated writing evaluation system (eRevise) as a fo…
    Computers and Education Open  ↗
  9. Deane (2019)
    The case for scenario-based assessment of written argumentation
    Reading and Writing  ↗
  10. Etxaniz, J., Azkune, G., Soroa, A., de Lacalle, O.L., & Artetxe, M. (2023). Do multilingual language models t…
     ↗
  11. Huawei (2023)
    A systematic review of automated writing evaluation systems
    Education and Information Technologies  ↗
  12. Klebanov, B.B., & Madnani, N. (2020). Automated evaluation of writing – 50 years and counting. In Proceedings…
     ↗
  13. Li (2014)
    The role of automated writing evaluation holistic scores in the ESL classroom
    System  ↗
  14. Lu (2024)
    Can ChatGPT effectively complement teacher assessment of undergraduate students’ academic…
    Assessment Evaluation in Higher Education  ↗
  15. Messick (1995)
    Validity of psychological assessment: Validation of inferences from persons' responses an…
    American Psychologist  ↗
  16. Mizumoto (2023)
    Exploring the potential of using an AI language model for automated essay scoring
    Research Methods in Applied Linguistics  ↗
  17. Olson (1976)
    On choosing a test statistic in multivariate analysis of variance
    Psychological Bulletin  ↗
  18. Palermo (2020)
    Implementing automated writing evaluation in different instructional contexts: A mixed-me…
    Journal of Writing Research  ↗
  19. Ramesh (2022)
    An automated essay scoring systems: a systematic literature review
    Artificial Intelligence Review  ↗
  20. Saglam (2025)
    Comparative evaluation of artificial intelligence models GPT-4 and GPT-3.5 in clinical de…
    BMC Medical Informatics and Decision Making  ↗
  21. Seßler, K., Fürstenberg, M., Bühler, B., & Kasneci, E. (2025). Can AI grade your essays? A comparative analys…
     ↗
  22. Shermis (2001)
    On-line grading of student essays: PEG goes on the World Wide Web
    Assessment Evaluation in Higher Education  ↗
  23. Steiss (2024)
    Comparing the quality of human and ChatGPT feedback of students’ writing
    Learning and Instruction  ↗
  24. Warschauer (2008)
    Automated writing assessment in the classroom
    Pedagogies: An International Journal  ↗
  25. Warschauer (2006)
    Automated writing evaluation: Defining the classroom research agenda
    Language Teaching Research  ↗
  26. Wetzler (2024)
    Grading the graders: comparing generative AI and human assessment in essay evaluation
    Teaching of Psychology  ↗
  27. Williamson (2012)
    A framework for evaluation and use of automated scoring
    Educational Measurement: Issues and Practice  ↗
  28. Wilson (2024)
    Recommendations for integrating automated writing evaluation with evidence-based instruct…
    International Journal of Changes in Education  ↗
  29. Wilson (2016)
    Automated essay evaluation software in english language arts classrooms: Effects on teach…
    Computers Education  ↗
  30. Wilson (2021)
    Automated feedback and automated scoring in the elementary grades: Usage, attitudes, and …
    International Journal of Artificial Intelligence in Education  ↗
  31. Wilson (2020)
    Automated writing evaluation and feedback: Multiple metrics of efficacy
    Journal of Educational Computing Research  ↗
  32. Wilson (2024)
    A latent dirichlet allocation approach to understanding students’ perceptions of automate…
    Computers and Education Open  ↗
  33. Wu, T.I., Lo, T.H., Chao, F.A., Sung, Y.T., & Chen, B. (2023). Effective neural modeling leveraging readabili…
     ↗
  34. Xiao, C., Ma, W., Song, Q., Xu, S.X., Zhang, K., Wang, Y., & Fu, Q. (2025). Human-AI collaborative essay scor…
     ↗
  35. Yang (2024)
    Learner interaction with, and response to, AI-programmed automated writing evaluation fee…
    Education and Information Technologies  ↗
  36. Zhai (2024)
    Multi-prompts learning with cross-modal alignment for attribute-based person re-identification
    Proceedings of the AAAI Conference on Artificial Intelligence  ↗
  37. Zupanc (2017)
    Automated essay evaluation with semantic analysis
    Knowledge-Based Systems  ↗
CrossRef global citation count: 0 View in citation network →