Abstract

The study reported in the paper starts with a hypothesis that errors observable in writing performances can account for much of the variability of the ratings awarded to them. The assertion is that this may be the case even when prescribed rating criteria explicitly direct rater focus towards successfully performed aspects of a writing performance rather than towards errors. The hypothesis is tested on a sample of texts rated independently of the study, using a five-point analytic rating scale involving ‘Can do’-like descriptors. The correlation between errors and ratings is ascertained using ordinal logistic regression, with Pseudo R2 of 0.51 discerned overall. Thus, with roughly 50% of score variability explainable by error occurrences, the stated hypothesis is considered confirmed. The study goes on to discuss the consequences of the findings and their potential employ in assessment of writing beyond the local assessment context.

Journal
Assessing Writing
Published
2024-01-01
DOI
10.1016/j.asw.2023.100806
CompPile
Search in CompPile ↗
Open Access
OA PDF Hybrid
Topics
Export

Citation Context

Cited by in this index (1)

  1. Assessing Writing

References (45) · 3 in this index

  1. Anaphoric reference: grammatical or pragmatic?
    Thought and Reference  
  2. Representative design and probabilistic theory in a functional psychology
    Psychological Review  
  3. Cesnik, H. (2019). Model logistic regression (BiOrd). [Brochure].
  4. Experience and learning
    An Introduction to Experiential Learning
  5. The impact of spelling errors on trained raters' scoring decisions
    Language Education & Assessment  
Show all 45 →
  1. The Prague School of linguistics and language teaching
    Journal of Linguistics  
  2. DeKeyser, R. (2008). The complexities of defining complexity. Paper presented at AAAL 2008, April 1st, Washin…
  3. A modern introduction to probability and statistics: Understanding why and how
  4. Towards a taxonomy of text features highly indicative of context-appropriate L2 writing c…
    System  
  5. Identifying errors in a learner corpus – the two stages of error location vs. error descr…
    Applied Corpus Linguistics  
  6. Quality measurements of error annotation - ensuring validity through reliability
    The European English Messenger
  7. Use of error profiles in applied linguistics – Empowering language instruction by catalog…
    Power in Language, culture, literature and education: The perspectives of English studies
  8. Towards an error taxonomy for student writing
    Zeitschrift für Interkulturellen Fremdsprachenunterricht
  9. Errors as indicators of writing task difficulty at the Slovene general Matura in English
    International Journal of Applied Linguistics  
  10. Detecting and measuring rater effects in performance assessments: Advances in many-facet …
    Festschrift in Honour of Günther Sigott: Advanced Methods in Language Testing
  11. Evaluating rater responses to an online training program for L2 writing assessment
    Language Testing  
  12. A tale of two models: Psychometric and cognitive perspectives on rater-mediated assessmen…
    Psychological Test and Assessment Modelling
  13. Learner Language and Language Learning
  14. Discovering Statistics using IBM SPSS
  15. One norm to rule them all? Corpus-derived norms in learner corpus research and foreign la…
    Language Teaching  
  16. Odds ratios and logistic regression: Further examples of their use and interpretation
    The Stata Journal  
  17. The link between vocabulary knowledge and spoken L2 fluency
    Language Learning Journal  
  18. Experiential Learning: Experience as the Source of Learning and Development
  19. Principles and Practice in Second Language Acquisition
  20. Effects of spelling errors on the perception of writers
    The Journal of General Psychology  
  21. Functional adequacy in L2 writing: Towards a new rating scale
    Language Testing  
  22. Error: Some problems of definition, identification, and distinction
    Applied Linguistics  
  23. Spelling skills of business students: An empirical investigation
    The Journal of Business Communication  
  24. A study of the decision–making behaviour of composition markers
    Performance Testing, Cognition and Assessment: Selected Papers from the 15th Language Testing Research Colloquium
  25. A handbook of reflective and experiential learning
    Theory and Practice
  26. A note on a general definition of the coefficient of determination
    Biometrika  
  27. Norušis, M. (2011). IBM SPSS Statistics 19. Guide to data analysis. Addison–Wesley.
  28. A Comprehensive Grammar of the English Language
  29. Validation of holistic scoring for ESL writing assessment: How raters evaluate compositions
    Fairness and validation in language assessment. Selected papers from the 19th Language Testing Colloquium
  30. The effect of written feedback types on students' academic texts: A pilot study
    AAA: Arbeiten aus Anglistik und Amerikanistik
  31. Refining the Scope – Substance error taxonomy: A closer look at Substance
    Corpora in Applied Linguistics – Current Approaches
  32. The Uses of Argument
  33. The think aloud method
    A Practical Guide to Modelling Cognitive Processes
  34. Shoot the messenger and disregard the message? Children’s attitudes toward spelling
    Reading Psychology  
  35. Pseudo–R2 measures for some common limited dependent variable models
    Sonderforschungsbereich
  36. Language Testing and Validation: An Evidence–based Approach
  37. Regression models with ordinal variables
    American Sociological Review  
  38. Assessing Writing
  39. Assessing Writing
  40. Assessing Writing