Assessing Writing Jul 2012

Using generalizability theory to examine the accuracy and validity of large-scale ESL writing assessment

Jinyan Huang
Journal
Assessing Writing
Published
2012-07-01
DOI
10.1016/j.asw.2011.12.003
CompPile
Search in CompPile ↗
Topics
assessment multilingual writers

Citation Context

Cited by in this index (8)

  1. Reed et al. (2026)
    Associations of adolescents’ argumentative writing scores and growth when evalu…
    Assessing Writing
  2. Naismith et al. (2024)
    The impact of task duration on the scoring of independent writing responses of …
    Assessing Writing
  3. Huang et al. (2023)
    Examining the reliability of an international Chinese proficiency standardized …
    Assessing Writing
  4. Elosua (2022)
    Validity evidences for scoring procedures of a writing assessment task. A case …
    Assessing Writing
  5. Li et al. (2022)
    The impact of essay organization and overall quality on the holistic scoring of…
    Assessing Writing
Show all 8 →
  1. Poe et al. (2019)
    Evidence of fairness: Twenty-five years of research in Assessing Writing
    Assessing Writing
  2. Ohta et al. (2018)
    Integrated writing scores based on holistic and multi-trait scales: A generaliz…
    Assessing Writing
  3. Janssen et al. (2015)
    Building a better rubric: Mixed methods rubric revision
    Assessing Writing

Cites in this index (5)

  1. Huang (2008)
    How accurate are ESL students’ holistic writing scores on large-scale assessmen…
    Assessing Writing
  2. Brown et al. (2004)
    Accuracy in the scoring of writing: Studies of reliability and validity using a…
    Assessing Writing
  3. Sudweeks et al. (2004)
    A comparison of generalizability theory and many-facet Rasch measurement in an …
    Assessing Writing
  4. Weigle (1999)
    Investigating rater/prompt interactions in writing assessment: Quantitative and…
    Assessing Writing
  5. Huot (1990)
    Reliability, Validity, and Holistic Scoring: What We Know and What We Need to Know
    College Composition and Communication
Also cites 33 works outside this index ↓
  1. Bachman (2000)
    Modern language testing at the turn of the century: Assuring that what we count counts
    Language Testing  ↗
  2. Bachman (1995)
    Investigating variability in tasks and rater judgment in a performance test of foreign speaking
    Language Testing  ↗
  3. Brown (1991)
    Do English and ESL faculties rate writing samples differently?
    TESOL Quarterly  ↗
  4. Brown (1984)
    A categorical instrument for scoring second language writing skills
    Language Learning  ↗
  5. Campbell (1959)
    Convergent and discriminant validation by the multitrait-multimethod method matrix
    Psychological Bulletin  ↗
  6. Connor-Linton (1995)
    Looking behind the curtain: What do L2 composition ratings really mean?
    TESOL Quarterly  ↗
  7. Cumming (2002)
    Decision-making while rating ESL/EFL writing tasks: A descriptive framework
    The Modern Language Journal  ↗
  8. Feldt (1965)
    The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty
    Psychometrika  ↗
  9. Feldt (1969)
    A test of the hypothesis that Cronbach's alpha or Kuder-Richardson coefficient twenty is …
    Psychometrika  ↗
  10. Gao (2001)
    Variability of estimated variance components and related statistics in a performance assessment
    Applied Measurement in Education  ↗
  11. Gamaroff (2000)
    Rating reliability in language assessment: The bug of all bears
    System  ↗
  12. Hakstian (1976)
    A k-sample significance test for independent alpha coefficients
    Psychometrika  ↗
  13. Hamp-Lyons (1994)
    Examining expert judgments of task difficulty on essay tests
    Journal of Second Language Writing  ↗
  14. Hinkel (2003)
    Simplicity without elegance: Features of sentences in L1 and L2 academic texts
    TESOL Quarterly  ↗
  15. Huang (2009)
    Factors affecting the assessment of ESL students’ writing
    International Journal of Applied Educational Studies  ↗
  16. Jenkins (2000)
    Cultural and linguistic miscues: A case study of international teaching assistant and aca…
    International Journal of Intercultural Relations  ↗
  17. Janopoulos (1992)
    University faculty tolerance of NS and NNS writing errors: A comparison
    Journal of Second Language Writing  ↗
  18. Johnson (2000)
    The relation between score resolution methods and interrater reliability: An empirical st…
    Applied Measurement in Education  ↗
  19. Kobayashi (1992)
    Native and nonnative reactions to ESL compositions
    TESOL Quarterly  ↗
  20. Kristof (1963)
    The statistical theory of stepped-up reliability coefficients when a test has been divide…
    Psychometrika  ↗
  21. Kraiger (1990)
    Generalizability theory as construct-related evidence of the validity of performance ratings
    Human Performance  ↗
  22. Lee (2007)
    Evaluating prototype tasks and alternative rating schemes for a new ESL writing test thro…
    International Journal of Testing  ↗
  23. Marcoulides (1989)
    Performance appraisal: Issues of validity
    Performance Improvement Quarterly  ↗
  24. Mendelsohn (1987)
    Professors’ ratings of language use and rhetorical organization in ESL compositions
    TESL Canada Journal  ↗
  25. Roberts (2008)
    Evaluating ESL: Making sense of university professors’ responses to second language writing
    Journal of Second Language Writing  ↗
  26. Santos (1988)
    Professors’ reactions to the writing of nonnative-speaking students
    TESOL Quarterly  ↗
  27. Schoonen (2005)
    Generalizability of writing scores: An application of structural equation modeling
    Language Testing  ↗
  28. Song (1996)
    Do English and ESL faculty differ in evaluating the essays of Native English-Speaking and…
    Journal of Second Language Writing  ↗
  29. Sweedler-Brown (1993)
    ESL essay evaluation: The influence of sentence-level and rhetorical features
    Journal of Second Language Writing  ↗
  30. Weigle (1994)
    Effects of training on raters of ESL compositions
    Language Testing  ↗
  31. Weigle (1998)
    Using FACETS to model rater training effects
    Language Testing  ↗
  32. Weigle (2003)
    Effects of task and rater background on the evaluation of ESL writing: A pilot study
    TESOL Quarterly  ↗
  33. Yamanaka (2005)
    Using generalizability theory in the evaluation of L2 writing
    JALT Journal  ↗
CrossRef global citation count: 35 View in citation network →

Related Articles

  1. Business and Professional Communication Quarterly Mar 2026
    Selections From the ABC 2025 Annual International Conference, Long Beach, California, USA: Classroom Activities for Teaching Career Readiness Skills in the Business Communication Classroom ↗
    Andrew Cavanaugh; D. Joel Whalen
    assessment writing across the curriculum multilingual writers
  2. Written Communication Feb 2026
    Reading Medium and Communicative Purpose in Writing: Effects on Pausing Behaviour and Text Quality, Controlling for Reading Comprehension and Executive Functions ↗
    Ángel Valenzuela; Cristian A. Rojas-Barahona; Ramón D. Castillo; Ladislao Salmerón
    assessment curriculum design multimodality multilingual writers revision
  3. Business and Professional Communication Quarterly Jan 2026
    Business Communication as Cultural Text: The Use of Student-Made Online Advertisements in Teaching Intercultural Communicative Competence ↗
    Saheeh Shafi
    assessment multimodality online writing instruction multilingual writers
  4. Business and Professional Communication Quarterly Dec 2025
    Selections From the ABC 2024 Annual International Conference, Tulsa, Oklahoma, USA: Dynamic Ideas for Teaching Skills for Working in Groups in the Business Communication Classroom ↗
    Andrew Cavanaugh
    assessment multilingual writers
  5. Assessing Writing Oct 2025
    Judgment accuracy in primary school EFL writing assessment: Do text characteristics matter? ↗
    Ruth Trüb; Jens Möller; Julian Lohmann; Thorben Jansen; Stefan D. Keller
    assessment multilingual writers