Assessing Writing Apr 2026 Open Access

Pursuing fair writing assessment: Halo effects in primary school foreign language writing in grade six

Ruth Trüb FHNW University of Applied Sciences and Arts ; Julian Lohmann ; Jens Möller ; Stefan D. Keller

Abstract

Assessing the writing competence of pupils learning English as a foreign language (EFL) at primary school is associated with specific challenges because of learners’ limited language resources. This study investigates the extent to which characteristics of their texts trigger so-called halo effects. Halo effects are an assessment bias where the quality of one feature unintentionally influences the evaluation of other aspects. The study examines halo effects across nine aspects of text quality (communicative effect, level of detail, coherence, cohesion, complexity of syntax and grammar, correctness of syntax and grammar, vocabulary, orthography and punctuation), based on a random sample of narrative texts from a sixth-grade corpus. 200 pre-service teachers assessed four randomly assigned texts. Halo effects were calculated by comparison to expert ratings using multi-level regression analyses. Results show that orthography and vocabulary were the two main triggers of halo effects. Punctuation also triggered some halo effects, but to a smaller extent. The assessment of communicative effect, complexity and correctness of syntax and grammar was not determined by the corresponding text quality but dominated by other criteria. Results highlight the importance of being aware of halo effects when assessing young EFL learners’ texts and emphasise the need for suitable training measures. • Analysis of halo effects across nine aspects of text quality. • Random sample of narrative texts from a sixth-grade EFL corpus. • Orthography and vocabulary are the two main triggers of halo effects. • Punctuation also triggers halo effects but to a smaller extent. • Halo effects call for awareness and targeted training.

Journal
Assessing Writing
Published
2026-04-01
DOI
10.1016/j.asw.2026.101036
CompPile
Search in CompPile ↗
Open Access
OA PDF Hybrid
Topics
assessment multilingual writers grammar and mechanics
Export
BibTeX RIS

Citation Context

Cited by in this index (0)

No articles in this index cite this work.

Cites in this index (5)

  1. Trüb et al. (2025)
    Judgment accuracy in primary school EFL writing assessment: Do text characteris…
    Assessing Writing
  2. Vögelin et al. (2019)
    The influence of lexical features on teacher judgements of ESL argumentative essays
    Assessing Writing
  3. Wolfe et al. (2016)
    Features of difficult-to-score essays
    Assessing Writing
  4. Harsch et al. (2012)
    Adapting CEF-descriptors for rating purposes: Validation by a combined rater tr…
    Assessing Writing
  5. Rezaei et al. (2010)
    Reliability and validity of rubrics for assessment through writing
    Assessing Writing
Also cites 22 works outside this index ↓
  1. Engelhard (1994)
    Examining rater errors in the assessment of written composition with a many-faceted Rasch model
    Journal of Educational Measurement  ↗
  2. Griva (2013)
    Writing skills and strategies of bilingual immigrant students learning Greek as a second …
    Reading & Writing  ↗
  3. Helmke (1987)
    Interactional effects of instructional quality and teacher judgement accuracy on achievement
    Teaching and Teacher Education  ↗
  4. Henik (1985)
    Control of halo error: A multiple regression approach
    Journal of Applied Psychology  ↗
  5. Jansen (2021)
    Don’t just judge the spelling! The influence of spelling on assessing second-language stu…
    Frontline Learning Research  ↗
  6. Lai (2015)
    Differentiation of illusory and true halo in writing scores
    Educational and Psychological Measurement  ↗
  7. Lindgren (2013)
    Interactional resources in the letters of young writers in Swedish and English
    Journal of Second Language Writing  ↗
  8. Lohmann (2025)
    Testing teacher judgment effects comprehensively: Accuracy, halo, frame of reference, str…
    Journal of Educational Psychology  ↗
  9. Lohmann (2025)
    A more comprehensive, more reliable multilevel approach for assessing and modeling teache…
    Educational Psychology Review  ↗
  10. Möller (2022)
    Judgment accuracy of German student texts: Do teacher experience and content knowledge matter?
    Teaching and Teacher Education  ↗
  11. Pajares (2003)
    Self-efficacy beliefs, motivation, and achievement in writing: A review of the literature
    Reading & Writing Quarterly  ↗
  12. Patekar (2021)
    A look into the practices and challenges of assessing young EFL learners’ writing in Croatia
    Language Testing  ↗
  13. Péry-Woodley (1991)
    Writing in L1 and L2: analysing and evaluating learners' texts
    Language Teaching  ↗
  14. Südkamp (2012)
    Accuracy of teachers’ judgments of students’ academic achievement: A meta-analysis
    Journal of Educational Psychology  ↗
  15. Sun (2021)
    Relationship between second language English writing self-efficacy and achievement: A met…
    Journal of Second Language Writing  ↗
  16. Tono (2014)
    Focus on learner writing at the beginning and intermediate stages
    International Journal of Corpus Linguistics  ↗
  17. Trüb, R. (2022). An empirical study of EFL writing at primary school. Narr Francke Attempto. doi:10.24053/978…
     ↗
  18. Vögelin (2018)
    The impact of vocabulary and spelling on judgments of ESL essays: An analysis of teacher …
    The Language Learning Journal  ↗
  19. Vögelin (2020)
    Organisational quality of ESL argumentative essays and its influence on pre-service teach…
    Cogent Education  ↗
  20. Vogt (2014)
    Assessment literacy of foreign language teachers: Findings of a European study
    Language Assessment Quarterly  ↗
  21. West (2011)
    The truth and bias model of judgment
    Psychological Review  ↗
  22. Zhu (2015)
    Teachers’ judgements of students’ foreign-language achievement
    European Journal of Psychology of Education  ↗
CrossRef global citation count: 0 View in citation network → Build reading path →

Related Articles

  1. Assessing Writing Apr 2026
    How do L2 writing subskills interact hierarchically? Insights from diagnostic classification models ↗
    Farshad Effatpanah; Hamdollah Ravand; Mahmoud Abdi Tabari; Yi-Hsin Chen; Olga Kunina-Habenicht
    assessment multilingual writers grammar and mechanics
  2. Assessing Writing Jan 2026
    The effects of online resource use on L2 learners’ computer-mediated writing processes and written products ↗
    Honglan Wang; Jookyoung Jung
    revision argument assessment digital rhetoric multilingual writers grammar and mechanics affect and writing
  3. Assessing Writing Jan 2026
    How reliable and valid is peer evaluation in adolescents’ L2 argumentative writing? ↗
    Albert W. Li; Steve Graham
    writing pedagogy teacher development argument assessment multilingual writers grammar and mechanics literacy studies
  4. Assessing Writing Oct 2025
    Assessing L2 writing formality using syntactic complexity indices: A fuzzy evaluation approach ↗
    Zhiyun Huang; Guangyao Chen; Zhanhao Jiang
    assessment multilingual writers grammar and mechanics
  5. Assessing Writing Oct 2025
    Judgment accuracy in primary school EFL writing assessment: Do text characteristics matter? ↗
    Ruth Trüb; Jens Möller; Julian Lohmann; Thorben Jansen; Stefan D. Keller
    teacher development assessment multilingual writers grammar and mechanics