Experienced but detached from reality: Theorizing and operationalizing the relationship between experience and rater effects

Iasonas Lamprianou University of Cyprus ; Dina Tsagari OsloMet – Oslo Metropolitan University ; Nansia Kyriakou Frederick University
Journal
Assessing Writing
Published
2023-04-01
DOI
10.1016/j.asw.2023.100713
CompPile
Search in CompPile ↗
Open Access
Closed
Export

Citation Context

Cited by in this index (0)

No articles in this index cite this work.

References (60) · 6 in this index

  1. A comparison of newly-trained and experienced raters on a standardized writing assessment
    Language Testing  
  2. What makes marking reliable? Experiments with UK examinations. Assessment in education
    Assessment in Education: Principles, Policy & Practice
  3. Individual differences in rater decision-making style: An exploratory mixed methods study
    Language Assessment Quarterly  
  4. Do ESL essay raters’ evaluation criteria change with experience? A mixed‐methods, cross‐s…
    TESOL Quarterly  
  5. Variability in ESL essay rating processes: The role of the rating scale and rater experience
    Language Assessment Quarterly  
Show all 60 →
  1. Effects of marking method and rater experience on ESL essay scores and rater performance
    Assessment in Education: Principles, Policy & Practice
  2. A many-facet Rasch analysis of the second language group oral discussion task
    Language Testing  
  3. Effect of genre on the generalizability of writing scores
    Language Testing  
  4. Application of an automated essay scoring engine to english writing assessment using many…
    Language Testing  
  5. Expertise in evaluating second language compositions
    Language Testing  
  6. The influence of training and experience on rater performance in scoring spoken language
    Language Testing  
  7. Examining Rater Effects in TestDaF Writing and Speaking Performance Assessments: A Many-F…
    Language Assessment Quarterly  
  8. Evaluating rater responses to an online training program for L2 writing assessment
    Language Testing  
  9. The effect of standardization-training on rater judgements for the IELTS writing module
    IELTS Collected Papers: Research in speaking and writing assessment
  10. Investigating rater severity/leniency in interpreter performance testing: A multifaceted …
    Interpreting International Journal of Research and Practice in Interpreting  
  11. Raters’ perceptions of rating scales criteria and its effect on the process and outcome o…
    Language Testing in Asia  
  12. Assessment in higher education: The potential for a community of practice to improve inte…
    Accounting Education  
  13. The temporal experience today: A phenomenological study
    The American Journal of Psychoanalysis  
  14. Validation of empirically derived rating scales for a story retelling speaking test
    Language Assessment Quarterly  
  15. Assessing Writing
  16. Three approaches to qualitative content analysis
    Qualitative Health Research  
  17. Evaluating CEFR rater performance through the analysis of spoken learner corpora
    Language Testing in Asia  
  18. The influence of holistic scoring procedures on reading and rating student essays
    Validating holistic scoring for writing assessment: Theoretical and empirical foundations
  19. Rater experience, rating scale length, and judgments of L2 pronunciation: Revisiting rese…
    Language Assessment Quarterly  
  20. Writing scale effects on raters: an exploratory study
    Language Testing in Asia  
  21. Examining rater accuracy and consistency with a special education observation protocol
    Studies in Educational Evaluation  
  22. Assessing Writing
  23. The stability of marker characteristics across tests of the same subject and across subjects
    Journal of Applied Measurement
  24. High stakes tests with self-selected essay questions: Addressing Issues of Fairness
    International Journal of Testing  
  25. Comparability of examination standards between subjects: An international perspective
    Oxford Review of Education  
  26. Unintended consequences of forced policy-making in high stakes examinations: The case of …
    Principles, Policy & Practice
  27. Investigation of rater effects using Social Network Analysis and Exponential Random Graph…
    Educational and Psychological Measurement  
  28. Measuring and visualizing coders’ reliability: New approaches and guidelines from experim…
    Sociological Methods and Research  
  29. Accuracyof measurement in the context of mathematics national curriculum tests in England…
    Journal of Educational Measurement  
  30. The longitudinal stability of rating characteristics in an EFL examination: Methodologica…
    Language Testing  
  31. A comparison of EFL raters’ essay-rating processes across two types of rating scales
    Language Assessment Quarterly  
  32. Assessing Writing
  33. The development and maintenance of rating quality in performance writing assessment: A lo…
    Language Testing  
  34. Assessing Writing
  35. Assessment criteria in a large-scale writing test: What do they really mean to the raters?
    Language Testing  
  36. Validity considerations ensuing from examinees’ perceptions about high-stakes national ex…
    Policy & Practice
  37. High-stakes language testing in the republic of Cyprus
  38. R Core Team. (2018). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Co…
  39. The effects of different rater training procedures on ESL essay raters’ rating accuracy
    Social Sciences and Humanities
  40. Robitzsch, A. , Kiefer, T. , & Wu, M. (2022-08-28). TAM: Test Analysis Modules (R package version 3.7–16). 〈h…
  41. Dichotomizing continuous predictors in multiple regression: A bad idea
    Statistics in Medicine  
  42. Do experience and text quality matter for raters’ decision-making behaviors?
    Language Testing  
  43. The assessment of writing ability: Expert readers versus lay readers
    Language Testing  
  44. Intervention in teachers’ differential scoring judgments in assessing L2 writing through …
    Studies in Educational Evaluation  
  45. The assessment of complex tasks: A double reading
    Studies in Higher Education  
  46. The effect of raters’ background and training on the reliability of direct writing tests
    The Modern Language Journal  
  47. Teachers as raters: Investigation of a long term writing assessment program
    L1 Educational Studies in Language and Literature  
  48. What goes through an examiner’s mind? Using verbal protocols to gain insights into the GC…
    British Educational Research Journal  
  49. Assessing Writing
  50. Temporal experience
    Psychological Bulletin  
  51. Using FACETS to model rater training effects
    Language Testing  
  52. Communities of practice: Learning, meaning and identity
  53. Willey, K., & Gardner, A. (2011). Getting tutors on the same page. AAEE Conference, Fremantle, Western Australia.
  54. Assessing Writing
  55. A methodologically improved study on raters’ personality and rating severity in writing a…
    SAGE Open