Assessing Writing Apr 2018 Open Access

Going online: The effect of mode of delivery on performances and perceptions on an English L2 writing test suite

Tineke Brunfaut Lancaster University ; Luke Harding Lancaster University ; Aaron Olaf Batty Keio University Shonan Fujisawa

Abstract

In response to changing stakeholder needs, large-scale language test providers have increasingly considered the feasibility of delivering paper-based examinations online. Evidence is required, however, to determine whether online delivery of writing tests results in changes to writing performance reflected in differential test scores across delivery modes, and whether test-takers hold favourable perceptions of online delivery. The current study aimed to determine the effect of delivery mode on the two writing tasks (reading-into-writing and extended writing) within the Trinity College London Integrated Skills in English (ISE) test suite across three proficiency levels (CEFR B1-C1). 283 test-takers (107 at ISE I/B1, 109 at ISE II/B2, and 67 at ISE III/C1) completed both writing tasks in paper-based and online mode. Test-takers also completed a questionnaire to gauge perceptions of the impact, usability and fairness of the delivery modes. Many-facet Rasch measurement (MFRM) analysis of scores revealed that delivery mode had no discernible effect, apart from the reading-into-writing task at ISE I, where the paper-based mode was slightly easier. Test-takers generally held more positive perceptions of the online delivery mode, although technical problems were reported. Findings are discussed with reference to the need for further research into interactions between delivery mode, task and level.

Journal
Assessing Writing
Published
2018-04-01
DOI
10.1016/j.asw.2018.02.003
CompPile
Search in CompPile ↗
Open Access
OA PDF Hybrid
Topics
technical communication multilingual writers editorial matter
Export
BibTeX RIS

Citation Context

Cited by in this index (3)

  1. Wang et al. (2024)
    Comparing Chinese L2 writing performance in paper-based and computer-based mode…
    Assessing Writing
  2. Liu et al. (2023)
    Comparing computer-based and paper-based rating modes in an English writing test
    Assessing Writing
  3. Poe et al. (2019)
    Evidence of fairness: Twenty-five years of research in Assessing Writing
    Assessing Writing

References (30) · 5 in this index

  1. Breetvelt (2009)
    Relations between writing processes and text quality: When and how?
    Cognition and Instruction  ↗
  2. Breland (2005)
    Comparability of TOEFL CBT essay prompts: Response-mode analyses
    Educational and Psychological Measurement  ↗
  3. Chen et al. (2011)
    Effects of computer versus paper administration of an adult functional writing …
    Assessing Writing
  4. Choi (2003)
    Comparability of a paper-based language test and a computer-based language test
    Language Testing  ↗
  5. Cumming (2014)
    Assessing integrated skills
Show all 30 →
  1. Davey (2011)
    Practical considerations in computer-based testing. ETS white paper
  2. Endres (2012)
    A comparability study of computer-based and paper-based writing tests
    Research Notes
  3. Fulcher (2003)
    Interface design in computer-based language testing
    Language Testing  ↗
  4. Jin (2010)
    A preliminary study of the validity of the internet-based CET-4: Factors affecting test t…
    Computer-Assisted Foreign Language Education
  5. Jin (2017)
    Computer literacy and the construct validity of a high-stakes computer-based writing assessment
    Language Assessment Quarterly  ↗
  6. Lee (2004)
    A comparative study of ESL writers’ performance in a paper-based and a computer…
    Assessing Writing
  7. Li (2006)
    The mediation of technology in ESL writing and its implications for writing ass…
    Assessing Writing
  8. Linacre, J.M. (2017). Facets computer program for many-facet Rasch measurement, version 3.80.0. Beaverton, Or…
  9. Ling (2017)
    Is writing performance related to keyboard type? An investigation from examinees’ perspec…
    Language Assessment Quarterly  ↗
  10. Maycock (2005)
    The effects on performance of computer familiarity and attitudes towards CB IELTS
    Research Notes
  11. McDonald (2002)
    The impact of individual differences on the equivalence of computer-based and paper-and-p…
    Computers & Education  ↗
  12. Noyes (2004)
    Paper-based versus computer-based assessment: Is workload another test mode effect?
    British Journal of Educational Technology  ↗
  13. Owston et al. (1992)
    The Effects of Word Processing on Students' Writing Quality and Revision Strategies
    Research in the Teaching of English
  14. Plakans et al. (2017)
    Exploring the relationship of organization and connection with scores in integr…
    Assessing Writing
  15. Powers (1994)
    Will they think less of my handwritten essay if others word process theirs? Effects on es…
    Journal of Educational Measurement  ↗
  16. Skehan (2009)
    Modelling second language performance: Integrating complexity, accuracy, fluency, and lexis
    Applied Linguistics  ↗
  17. Stoynoff (2013)
    Fairness in language assessment
    The encyclopedia of applied linguistics
  18. Taylor (1999)
    Examining the relationship between computer familiarity and performance on computer-based…
    Language Learning  ↗
  19. Trinity College London (2016)
    Integrated skills in english (ISE). specifications –Reading & writing. ISE foundation to ISE III
  20. Van Waes (2003)
    Writing profiles: The effect of the writing mode on pausing and revision patterns of expe…
    Journal of Pragmatics  ↗
  21. Weir (2005)
    Language testing and validation: An evidenced-based approach
  22. Wolfe (2005)
    An investigation of the impact of composition medium on the quality of scores from the TOEFL writing section: A report from the broad-based study. TOEFL Research Report RR-72
  23. Yu (2004)
    Investigating differences in examinee performance between computer-based and handwritten essays ETS Research Report RR-04-18
  24. Yu (2010)
    Effects of presentation mode and computer familiarity on summarization of extended texts
    Language Assessment Quarterly  ↗
  25. Zou (2016)
    Effects of test media on different EFL test-takers in writing scores and in the cognitive…
    Pedagogy and Education  ↗
CrossRef global citation count: 23 View in citation network → Build reading path →

Related Articles

  1. Technical Communication Quarterly Oct 2024
    Editors’ Use of Comprehensive Style Guides: The Case of Singular They ↗
    Jo Mackiewicz; Shaya Kraut; Allison Durazzi
    rhetorical criticism discourse analysis first-year composition writing pedagogy graduate education teacher development writing centers technical communication professional writing digital rhetoric multilingual writers grammar and mechanics literacy studies race and writing public rhetoric editorial matter
  2. Journal of Business and Technical Communication Jul 2010
    Peer Reviewing Across the Atlantic: Patterns and Trends in L1 and L2 Comments Made in an Asynchronous Online Collaborative Learning Exchange Between Technical Communication Students in Sweden and in the United States ↗
    Paul Anderson; Becky Bergman; Linda Bradley; Magnus Gustafsson; Aurora Matzke
    collaborative writing technical communication online writing instruction multilingual writers editorial matter
  3. Pedagogy Oct 2025
    Contributors ↗
    modern rhetorical theory rhetorical criticism genre theory cultural rhetorics first-year composition writing pedagogy advanced composition creative writing writing across the curriculum graduate education two-year college service learning teacher development technical communication professional writing labor and working conditions archival research multimodality artificial intelligence literacy studies race and writing gender and writing disability studies literary studies editorial matter
  4. The Peer Review Sep 2025
    Moving Against the Grain: Combining Writing Center Theory and In-House Editing Services to Create a Graduate Writing Center ↗
    Brian Harrell; Brook Wyers; Craig Theissen
    rhetorical criticism composition theory writing pedagogy graduate education teacher development collaborative writing writing centers peer tutoring professional writing labor and working conditions online writing instruction multilingual writers grammar and mechanics editorial matter
  5. Philosophy & Rhetoric Apr 2025
    Aristotle: Art of Rhetoric ↗
    Peter A. O’Connell
    classical rhetoric modern rhetorical theory rhetorical criticism revision argument multilingual writers grammar and mechanics public rhetoric editorial matter