Abstract

Assessing the writing competence of pupils learning English as a foreign language (EFL) at primary school is associated with specific challenges because of learners’ limited language resources. This study investigates the extent to which characteristics of their texts trigger so-called halo effects. Halo effects are an assessment bias where the quality of one feature unintentionally influences the evaluation of other aspects. The study examines halo effects across nine aspects of text quality (communicative effect, level of detail, coherence, cohesion, complexity of syntax and grammar, correctness of syntax and grammar, vocabulary, orthography and punctuation), based on a random sample of narrative texts from a sixth-grade corpus. 200 pre-service teachers assessed four randomly assigned texts. Halo effects were calculated by comparison to expert ratings using multi-level regression analyses. Results show that orthography and vocabulary were the two main triggers of halo effects. Punctuation also triggered some halo effects, but to a smaller extent. The assessment of communicative effect, complexity and correctness of syntax and grammar was not determined by the corresponding text quality but dominated by other criteria. Results highlight the importance of being aware of halo effects when assessing young EFL learners’ texts and emphasise the need for suitable training measures. • Analysis of halo effects across nine aspects of text quality. • Random sample of narrative texts from a sixth-grade EFL corpus. • Orthography and vocabulary are the two main triggers of halo effects. • Punctuation also triggers halo effects but to a smaller extent. • Halo effects call for awareness and targeted training.

Journal
Assessing Writing
Published
2026-04-01
DOI
10.1016/j.asw.2026.101036
CompPile
Open Access
OA PDF Hybrid
Topics
Export

Citation Context

Cited by in this index (0)

No articles in this index cite this work.

References (52) · 5 in this index

  1. Text – Sorten – Kompetenz. Eine echte Longitudinalstudie zur Entwicklung der Textkompetenz im Grundschulalter [Text – Genre – Competence. A true longitudinal study on the development of text competence at primary school age]
  2. Detecting halo effects across rubric criteria in L2 writing assessment: A many-facet Rasc…
    Applied Research on English Language
  3. Linguistische Textanalyse: Eine Einführung in Grundbegriffe und Methoden [Linguistic text analysis: An introduction to basic concepts and methods]
  4. Brock, R. (2015). Handbuch Writing (2014 und 2015). Kommentierte Schreibperformanzen 2014 und 2015: Gesamtbeu…
  5. Deutschschweizer Erziehungsdirektoren-Konferenz. (2016). Lehrplan 21 – von der D-EDK Plenarversammlung am 31.…
Show all 52 →
  1. Introduction to many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments
  2. Examining rater errors in the assessment of written composition with a many-faceted Rasch model
    Journal of Educational Measurement  
  3. Writing skills and strategies of bilingual immigrant students learning Greek as a second …
    Reading & Writing  
  4. A study of FL composing process and writing strategies employed by young learners
    Early learning of modern foreign languages. Processes and outcomes
  5. Typen und Funktionen der Evaluation von Schreibkompetenzen [Types and functions of writin…
    Schreibkompetenzen in der Fremdsprache. Aufgabengestaltung, kriterien-orientierte Bewertung und Feedback
  6. Hallet, W. (2016). Genres im fremdsprachlichen und bilingualen Unterricht. Formen und Muster der sprachlichen…
  7. Assessing Writing
  8. Hasselgreen, A., Carlsen, C., & Helness, H. (2004). European survey of language testing and assessment needs.…
  9. Equinox
    Assessing the Language of Young learners
  10. Assessment of young learner literacy linked to the Common European Framework of Reference for Languages
  11. Interactional effects of instructional quality and teacher judgement accuracy on achievement
    Teaching and Teacher Education  
  12. Hemmerich, W. (2016). StatistikGuru: Rechner zur Adjustierung des α-Niveaus [StatistikGuru: Calculator for ad…
  13. Control of halo error: A multiple regression approach
    Journal of Applied Psychology  
  14. A simple sequentially rejective multiple test procedure
    Scandinavian Journal of Statistics
  15. Genre and second language writing
  16. A closer look at halo error in performance ratings
    The Academy of Management Journal
  17. Don’t just judge the spelling! The influence of spelling on assessing second-language stu…
    Frontline Learning Research  
  18. Konrad, E., Holzknecht, F., Schwarz, V., & Spöttl, C. (2018). Assessing writing at lower levels: Research fin…
  19. Lai, E.R., Wolfe, E.W., & Vickers, D.H. (2012). Halo effects and analytic scoring. A summary of two empirical…
  20. Differentiation of illusory and true halo in writing scores
    Educational and Psychological Measurement  
  21. Towards a lexical view of language – a challenge for teachers
    Babylonia
  22. LimeSurvey: An Open Source survey tool
  23. Linacre, J.M. (2025). A user’s guide to FACETS: Rasch-model computer programs [Software manual]. 〈https://www…
  24. Interactional resources in the letters of young writers in Swedish and English
    Journal of Second Language Writing  
  25. Testing teacher judgment effects comprehensively: Accuracy, halo, frame of reference, str…
    Journal of Educational Psychology  
  26. A more comprehensive, more reliable multilevel approach for assessing and modeling teache…
    Educational Psychology Review  
  27. Judgment accuracy of German student texts: Do teacher experience and content knowledge matter?
    Teaching and Teacher Education  
  28. Detecting and measuring rater effects using many-facet Rasch measurement: Part II
    Journal of Applied Measurement
  29. Self-efficacy beliefs, motivation, and achievement in writing: A review of the literature
    Reading & Writing Quarterly  
  30. A look into the practices and challenges of assessing young EFL learners’ writing in Croatia
    Language Testing  
  31. Writing in L1 and L2: analysing and evaluating learners' texts
    Language Teaching  
  32. Assessing Writing
  33. Accuracy of teachers’ judgments of students’ academic achievement: A meta-analysis
    Journal of Educational Psychology  
  34. Relationship between second language English writing self-efficacy and achievement: A met…
    Journal of Second Language Writing  
  35. Collated representative samples of descriptors of language competences developed for young learners aged 7–10 years
  36. Collated representative samples of descriptors of language competences developed for young learners aged 11–15 years
  37. Focus on learner writing at the beginning and intermediate stages
    International Journal of Corpus Linguistics  
  38. Assessing Writing
  39. Trüb, R. (2022). An empirical study of EFL writing at primary school. Narr Francke Attempto. doi:10.24053/978…
  40. UNESCO Institute for Statistics. (2012). International standard classification of education. ISCED 2011. 〈htt…
  41. The impact of vocabulary and spelling on judgments of ESL essays: An analysis of teacher …
    The Language Learning Journal  
  42. Assessing Writing
  43. Organisational quality of ESL argumentative essays and its influence on pre-service teach…
    Cogent Education  
  44. Assessment literacy of foreign language teachers: Findings of a European study
    Language Assessment Quarterly  
  45. The truth and bias model of judgment
    Psychological Review  
  46. Assessing Writing
  47. Teachers’ judgements of students’ foreign-language achievement
    European Journal of Psychology of Education