Effects of a genre and topic knowledge activation device on a standardized writing test performance

Natalia Ávila Reyes Pontificia Universidad Católica de Chile ; Diego Carrasco Pontificia Universidad Católica de Chile ; Rosario Escribano Pontificia Universidad Católica de Chile ; María Jesús Espinosa Diego Portales University ; Javiera Figueroa Alberto Hurtado University ; Carolina Castillo Pontificia Universidad Católica de Chile

Abstract

The aim of this article was twofold: first, to introduce a design for a writing test intended for application in large-scale assessments of writing, and second, to experimentally examine the effects of employing a device for activating prior knowledge of topic and genre as a means of controlling construct-irrelevant variance and enhancing validity. An authentic, situated writing task was devised, offering students a communicative purpose and a defined audience. Two devices were utilized for the cognitive activation of topic and genre knowledge: an infographic and a genre model. The participants in this study were 162 fifth-grade students from Santiago de Chile, with 78 students assigned to the experimental condition (with activation device) and 84 students assigned to the control condition (without activation device). The results demonstrate that the odds of presenting good writing ability are higher for students who were part of the experimental group, even when controlling for text transcription ability, considered a predictor of writing. These findings hold implications for the development of large-scale tests of writing guided by principles of educational and social justice. • Genre and topic knowledge are forms of prior knowledge relevant to writing. • Higher odds for better writing in students exposed to prior knowledge activation. • Results support use of prior knowledge activation in standardized assessment.

Journal
Assessing Writing
Published
2024-10-01
DOI
10.1016/j.asw.2024.100898
CompPile
Search in CompPile ↗
Open Access
OA PDF Hybrid
Topics
Export

Citation Context

Cited by in this index (0)

No articles in this index cite this work.

References (59) · 10 in this index

  1. The effects of choice versus preference on writing and the mediating role of perceived co…
    Journal of Educational Psychology  
  2. Standards for educational and psychological testing
  3. Understanding the lifelong journey of writing development
    Infancia York Aprendizaje  
  4. Genre in a changing world
  5. Scaffolding Students’ Writing Processes Through Dialogic Assessment
    Journal of Adolescent Adult Literacy  
Show all 59 →
  1. The psychology of written composition
  2. Lower-level developmental skills in beginning writing
    Reading and Writing  
  3. Writing in primary school
    Handbook of research on writing
  4. Research in the Teaching of English
  5. Castillo, C. & Ávila Reyes, N. (In Review). Students' sociodemographic characteristics and writing performanc…
  6. Cognitive validity evidence of computer- and paper-based writing tests and differences in…
    Assessing Writing
  7. Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessmen…
    Psychological Assessment  
  8. A multi-dimensional examination of adolescent writing: Considering the writer, genre and …
    Reading and Writing  
  9. Journal of Writing Analytics
  10. Research in the Teaching of English
  11. Assessing writing
  12. Written Communication
  13. Authentic Literacy Activities for Developing Comprehension and Writing
    The Reading Teacher  
  14. Monitoring raters in performance assessments
    Large-scale assessment programs for all students
  15. Effects of instruction in narrative structure on children's writing
    Journal of Educational Psychology  
  16. Promoting Fairness in EFL Writing Assessment: Are There Any Effects of the Writers’ Aware…
    Journal of Asia TEFL
  17. A writer(s)-within-community model of writing
    The Lifespan Development of Writing
  18. The roles of writing knowledge, motivation, strategic behaviors, and skills in predicting…
    The Elementary School Journal  
  19. Writing Skills, Knowledge, Motivation, and Strategic Behavior Predict Students’ Persuasiv…
    The Elementary School Journal  
  20. Examining the impacts of ignoring rater effects in mixed-format tests
    Journal of Educational Measurement  
  21. Assessing Writing
  22. The comparison of oral and written productions in two discourse types.
    Discourse Processes  
  23. Antiracist Writing Assessment Ecologies: Teaching and Assessing Writing for a Socially Just Future
  24. Effects of topic knowledge on the allocation of processing time and cognitive effort to w…
    Memory Cognition  
  25. Validity and fairness implications of varying time conditions on a diagnostic test of aca…
    SYSTEM  
  26. Children reading and writing: Structures and strategies
  27. Effect size measures for multilevel models: definition, interpretation, and TIMSS example…
    Assessments in Education
  28. Assessment criteria in a large-scale writing test: what do they really mean to the raters?
    Language Testing  
  29. Validez de evaluaciones educacionales en Chile y Latinoamérica
  30. Domain knowledge and linguistic knowledge in the development of writing ability
    Journal of Memory and Language  
  31. Children's discourse skill: Form and modality requirements of schooled writing
    Discourse Processes  
  32. Journal of Writing Research
  33. Revising for meaning: Effects of knowledge and strategy
    Journal of Educational Psychology  
  34. Muthén, L.K., & Muthén, B.O. (2017). Mplus User’s Guide (8th ed.). Muthén & Muthén.
  35. The relationship between the discourse knowledge and the writing performance of elementar…
    Journal of Educational Psychology  
  36. The relationship of discourse and topic knowledge to fifth graders’ writing performance
    Journal of Educational Psychology  
  37. Towards a new NAPLAN: Testing to the teaching
  38. Assessing Writing
  39. College English
  40. Fostering topic knowledge: Essential for academic writing
    Reading and Writing  
  41. Learning to read and write genre- specific text: Roles of authentic experience and explic…
    Reading Research Quarterly International Reading Association  
  42. Our validity looks like justice. Does yours?
    Language Testing  
  43. High-Quality Formative Writing Assessment for Middle School Students in Tier 2 Literacy I…
    Learning Disabilities Research Practice  
  44. Assessing Writing
  45. A Study of Reliability Across Multiple Raters When Using the NAEP and MDFS Rubrics to Mea…
    Reading Psychology  
  46. A study of reliability across multiple raters when using the NAEP and MDFS rubrics to mea…
    Reading Psychology  
  47. Social justice and educational measurement: John Rawls, the history of testing, and the future of education
  48. The effects of topic familiarity on emotionality and linguistic complexity in EAP writing
    Language Teaching Research
  49. Usar la lengua escrita en la escuela
    Revista Iberoamericana Délelőtt EducacióN  
  50. Written Communication
  51. Structural Equation Modeling: Applications Using Mplus
  52. The roles of writing knowledge, motivation, strategic behaviors, and skills in predicting…
    Reading and Writing  
  53. Rater Connections and the Detection of Bias in Performance Assessment
    Measurement
  54. The impacts of characteristics of disconnected subsets on group anchoring in incomplete r…
    Psychological Test and Assessment Modeling