Automated essay scoring: Psychometric guidelines and practices

Chaitanya Ramineni Educational Testing Service ; David M. Williamson Educational Testing Service
Journal
Assessing Writing
Published
2013-01-01
DOI
10.1016/j.asw.2012.10.004
CompPile
Search in CompPile ↗
Open Access
Closed
Export

Citation Context

References (74) · 7 in this index

  1. Responding to and assessing student writing: The uses and limits of technology
    Teaching writing with computers
  2. Modified CVA features (unpublished report)
  3. Performance of a generic approach in automated essay scoring
    Journal of Technology Learning, and Assessment
  4. Automated essay scoring with e-rater v.2
    Journal of Technology Learning, and Assessment
  5. Moving the field forward: Some thoughts on validity and automated scoring
    Automated scoring of complex-tasks in computer-based testing
Show all 74 →
  1. Automated scoring of constructed-response literacy and mathematics items.
  2. Validity and automated scoring: It's not only the scoring
    Educational Measurement: Issues and Practice  
  3. Two experiments on automatic scoring of spoken language proficiency
    Proceedings of InSTIL2000 (Integrating Speech Tech. in Learning)
  4. Understanding scoring reliability: Experiments in calibrating essay readers
    Journal of Educational Statistics  
  5. Rule-based methods for automatic scoring: Application in a licensing context
    Automated scoring for complex constructed response tasks in computer based testing
  6. New SAT writing prompt study: Analyses of group impact and reliability. (Research Report 2004-1)
  7. Comparison of human and machine scoring of essays: Differences by gender, ethnicity, and …
    Applied Measurement in Education  
  8. The e-rater® scoring engine: Automated essay scoring with natural language processing
    Automated essay scoring: A cross-disciplinary perspective
  9. CAA of short non-MCQ answers
    Proceedings of the 5th International CAA Conference
  10. Speech interaction with Saybot player, a CALL software to help Chinese learners of English
    Proceedings of The International Speech Communication Association Special Interest Group on Speech and Language Technology in Education (SLaTE)
  11. Automated scoring technologies and the rising influence of error
    English Journal  
  12. Validity issues for performance-based tests scored with computer-automated scoring systems
    Applied Measurement in Education  
  13. Statistical power analysis for the behavioral sciences
  14. Conference on College Composition and Communication (2004). CCCC position statement on teaching, learning, an…
  15. Decision-making while rating ESL/EFL writing tasks: A descriptive framework
    Modern Language Journal  
  16. Assessing Writing
  17. Considerations in the development of accounting simulations
    Paper presented at the meeting of the National Council on Measurement in Education
  18. An overview of automated scoring of essays
    Journal of Technology Learning and Assessment
  19. On a scale: A social history of writing assessment in America
  20. The equivalence of weighted kappa and the intraclass correlation coefficient as measures …
    Educational and Psychological Measurement  
  21. The SRI EduSpeak™ system: Recognition and pronunciation scoring for language learning
    Proceedings of InSTILL (Integrating Speech Technology in Language Learning)
  22. The effect of the quality of preceding responses on the grades assigned to subsequent res…
    Journal of Educational Measurement  
  23. Assessing Writing
  24. College English
  25. The use of model essays to reduce context effects in essay scoring
    Journal of Educational Measurement  
  26. (Re) Articulating writing assessment in teaching and learning
  27. Writing assessment: A techno-history
    Handbook of writing research
  28. Automated scoring and annotation of essays with the Intelligent Essay Assessor
    Automated essay scoring: A cross-disciplinary perspective
  29. C-rater: Scoring of short-answer questions
    Computers and the Humanities  
  30. Rater effects on essay scoring: A multilevel analysis of severity drift, central tendency…
    Journal of Educational Measurement  
  31. Measuring the impact of judge severity on examination scores
    Applied Measurement in Education  
  32. A regression-based procedure for automated scoring of a complex medical performance assessment
    Automated scoring of complex tasks in computer based testing
  33. Towards robust computerized marking of free-text responses
    Proceedings of the Sixth International Computer Assisted Assessment Conference
  34. Common Core State Standards
  35. The imminence of grading essays by computer
    Phi Delta Kappan
  36. The use of the computer in analyzing student essays
    International Review of Education  
  37. Project essay grade: PEG
    Automated essay scoring: A cross-disciplinary perspective
  38. Pearson (2009, March). PTE academic automated scoring. Retrieved from http://www.pearsonpte.com/SiteCollectio…
  39. Comparing the validity of automated and human scoring of essays
    Educational Computing Research  
  40. Stumping e-rater: Challenging the validity of automated essay scoring
    Computers in Human Behavior  
  41. Evaluating the construct-coverage of the e-rater® scoring engine (Research Report 09-01)
  42. Assessing Writing
  43. Evaluation of e-rater® for GRE Issue and Argument prompts (Research Report 12-02)
  44. Evaluation of e-rater® scoring engine for TOEFL Independent and Integrated prompts (Research Report 12-06)
  45. Testing and assessing mathematical skills by a script based system
    Paper presented at the 10th International Conference on Interactive Computer aided Learning
  46. Reducing errors due to the use of judges
    Practical assessment, Research and Evaluation
  47. An evaluation of IntelliMetric™ essay scoring system
    The Journal of Technology, Learning and Assessment
  48. Rating the ratings: Assessing the psychometric quality of rating data
    Psychological Bulletin  
  49. A human–computer collaborative approach to the marking of free text answers
    Proceedings of the 8th International CAA Conference
  50. Automated essay scoring: A cross-disciplinary perspective
  51. Applications of computers in assessment and analysis of writing
    Handbook of writing research
  52. How important is content in the ratings of essay assessments?
    Assessment in Education: Principles, Policy and Practice  
  53. Validation and extension of the mathematical expression response type: Applications of schema theory to automatic scoring and item generation in mathematics (GRE Board Professional Report 93-24P)
  54. Creating writers through 6-trait Writing Assessment and Instruction
  55. The problem of the English examination
    Educational Record
  56. Information extraction and machine learning: Auto-marking short free text responses to sc…
    Proceedings of the 12th International Conference on Artificial Intelligence in Education (AIED)
  57. An overview of current research on automated essay grading
    Journal of Information Technology Education  
  58. Holistic assessment: What goes on in the rater's mind?
    Assessing second language writing in academic contexts
  59. Automated writing evaluation: Defining the classroom research agenda
    Language Teaching Research  
  60. Assessing Writing
  61. Teaching and assessing writing: Recent advances in understanding, evaluating, and improving student performance
  62. ’Mental model’ comparison of automated and human scoring
    Journal of Educational Measurement  
  63. Automated Scoring for the Assessment of Common Core Standards
  64. A framework for evaluation and use of automated scoring
    Educational Measurement: Issues and Practices  
  65. Assessing Writing
  66. Written Communication
  67. Automated scoring of spontaneous speech using SpeechRater v1.0 (Research Report 08-62)
  68. A review of strategies for validating computer-automated scoring
    Applied Measurement in Education  
  69. Towards automatic scoring of non-native spontaneous speech
    Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL