Abstract

The assessment of students' writing skills through essays is a common practice in educational institutions. Scoring of essays requires considerable judgment on the part of those who rate the response. When raters assign different scores to an essay, testing practitioners must resolve the discrepancy before computing an operational score to report to the examinee. This study investigated five forms of score resolution that were reported in a national survey of state department of education-testing agencies. The study examined the effect that each form of resolution has on the reliability of the resulting operational scores. It is shown that some methods of resolution can be associated with higher interrater reliability than can others. It is also shown that the choice of resolution can affect the magnitude of the reported score as well as the final passing rate of an assessment.

Journal
Written Communication
Published
2001-04-01
DOI
10.1177/0741088301018002003
CompPile
Search in CompPile ↗
Open Access
Closed
Topics
Export

Citation Context

References (30) · 5 in this index

  1. The direct assessment of writing skill: A measurement review
  2. Technical issues in large-scale performance assessment
  3. Written Communication
  4. Validating holistic scoring for writing assessment: Theoretical and empirical foundations
  5. 10.1111/j.1745-3984.1999.tb00544.x
Show all 30 →
  1. Educational measurement
  2. Generalizability analysis for educational assessments
  3. 10.1177/0013164497057003001
  4. Development of writing
  5. Written Communication
  6. Written Communication
  7. A practical guide to alternative assessment
  8. Handbook for focused holistic scoring
  9. 10.3102/00346543060002237
  10. Written Communication
  11. 10.1177/109821400002100106
  12. 10.1207/S15324818AME1302_1
  13. annual meeting of the American Educational Research Association
  14. Window on the World of Family Literacy
  15. 10.1111/j.1745-3992.1999.tb00010.x
  16. Measurement and evaluation in education and psychology
  17. annual meeting of the American Educational Research Association
  18. Profile of 1994-1995 state assessment systems and reported results
  19. Annual survey of state student assessment programs: A summary report
  20. Generalizability theory: A primer
  21. Validating holistic scoring for writing assessment: Theoretical and empirical foundations
  22. Research in the Teaching of English
  23. annual meeting of the American Educational Research Association
  24. Validating holistic scoring for writing assessment: Theoretical and empirical foundations
  25. An overview of writing assessment: Theory, research, and practice