Assessing Writing

31 articles
Year: Topic: Clear
Export:
argument ×

April 2026

  1. Associations of adolescents’ argumentative writing scores and growth when evaluated by different human raters and artificial intelligence models
    doi:10.1016/j.asw.2026.101015

January 2026

  1. The effects of online resource use on L2 learners’ computer-mediated writing processes and written products
    Abstract

    While previous studies on online resource use in L2 writing have focused on the overall writing quality, limited attention has been paid to its effects on linguistic complexity and real-time writing processes. Addressing this gap, the present study explored how online resource use influences both the processes and products of L2 writing. Forty-nine intermediate L2 learners completed two computer-mediated argumentative writing tasks, either with or without the use of online resources. Writing behaviors were captured via keystroke logging and screen recording, and analyzed for search activity, fluency, pausing, and revision quantity. Cognitive processes were examined through stimulated recall interviews, and written products were evaluated for both quality and linguistic complexity. The results showed that participants spent an average of 14 % of task time using online resources, with considerable individual variation. Mixed-effects modeling revealed that resource use facilitated the production of more sophisticated words, with marginal influence on writing quality or syntactic complexity. Resource use was also associated with longer between-word pauses, fewer within-word pauses, and reduced revisions. These findings highlight the potential of online resource use to enhance the authenticity of L2 writing assessment tasks without compromising test validity, while encouraging the use of more advanced vocabulary in writing. • Learners spent 14 % of the total writing task time using online resources. • Online resource use had no significant impact on L2 writing quality. • Online resource use improved lexical sophistication, not syntactic complexity. • Online resource use reduced within-word pauses and aided spelling retrieval. • Online resource use led to fewer revisions but did not affect fluency.

    doi:10.1016/j.asw.2025.100994
  2. How reliable and valid is peer evaluation in adolescents’ L2 argumentative writing?
    Abstract

    Peer evaluation is widely recognized for its educational benefits; however, its reliability and validity, particularly among adolescent second-language (L2) writers at the early stages of English language and literacy development, remain insufficiently explored. This explanatory sequential mixed-methods study investigated the reliability and validity of peer evaluation in English argumentative writing among 35 Grade 10 and 37 Grade 12 students from a public high school in Beijing, China. Twelve of the participating students (six at each grade) were interviewed about the validity, reliability, and value of peer evaluation. The findings indicated that peer evaluations demonstrated high levels of reliability and validity, with peer-assessed writing scores closely aligning with inter-teacher assessments. Notably, variations were observed among Grade 10 students, particularly in the evaluation of lower-order writing skills, such as grammar and vocabulary, which exhibited reduced validity. These results underscore the potential of peer evaluation in assessing higher-order content-level writing across varying levels of L2 English writing proficiency. The study also highlights areas where adolescent L2 writers may require additional support to enhance the effectiveness of peer evaluation practices in English argumentative writing. Implications for improving English argumentative writing instruction and refining peer evaluation strategies in high school L2 English classrooms are discussed. • Peer evaluation shows high reliability, similar to inter-teacher rating. • Peer evaluation works well for higher-order skills in L2 argumentative writing. • 10th graders struggled with evaluating lower-order skills like grammar. • 12th graders evaluate lower- and higher-order skills with greater validity than 10th graders.

    doi:10.1016/j.asw.2025.100992
  3. Is it beneficial to strive for perfection in writing?: Exploring the relationship between perfectionism, motivational regulation, and second language (L2) writing performance
    Abstract

    Perfectionism, a personality trait characterized by the pursuit of flawlessness and high personal standards, and motivational regulation, the strategies through which individuals manage their motivational states, have received limited attention in second language (L2) writing. Framed within social cognitive theory, this study examines how two dimensions of perfectionism—perfectionistic strivings and perfectionistic concerns—relate to writing performance (syntactic complexity, accuracy, lexical complexity, and fluency) and how motivational regulation sub-strategies (interest enhancement, self-talk, and emotional control) mediate these relationships. Data from 689 university students in China were analyzed using questionnaires and argumentative writing samples. Results indicated that perfectionistic strivings positively predicted syntactic complexity, accuracy, and lexical complexity, while perfectionistic concerns negatively predicted these dimensions; neither dimension significantly affected fluency. Crucially, motivational regulation sub-strategies partially mediated the relations between perfectionism and writing performance. These findings underscore the importance of distinguishing perfectionism dimensions and targeting motivational regulation strategies to improve L2 writing. Implications for instruction and directions for future longitudinal research are discussed. • Perfectionistic strivings and concerns affect writing via motivational regulation. • Strivings improve syntax, accuracy, and lexical complexity; concerns hinder them. • Most motivational regulation sub-strategies mediate perfectionism’s impact on CALF. • Perfectionism influences writing through motivational regulation.

    doi:10.1016/j.asw.2025.101012

October 2025

  1. Understanding the critical thinking experiences of L2 student writers engaged in linguistically supported peer feedback giving
    doi:10.1016/j.asw.2025.100977

April 2025

  1. The influence of working memory and proficiency on phraseological growth: A longitudinal study of adjective-noun combinations in Chinese EFL learners’ argumentative writing
    doi:10.1016/j.asw.2025.100915
  2. Designing a rating scale for an integrated reading-writing test: A needs-oriented approach
    Abstract

    To meet the current trends in higher education, there is accountability on EAP programmes to prepare and assess students’ access to higher education. Thus, multimodal tasks including integrated writing (IW) assessments have seen a resurgence because they arguably closely mirror academic writing. However, test practicality constraints and variability in the use and format of these assessments mean rating scales often fall short in substantiating the central claims of IW assessment. We developed an integrated reading-writing scale taking into account reading-writing requirements and empirical research on IW tests designed to assess readiness for first-year humanities and social science courses. We approached test development as part of the ongoing validation efforts, detailing the considerations involved in the scale development process. We argue that alignment with academic writing requirements should guide the development of IW tests, thereby acknowledging and comprehending nuances of academic writing. The paper demonstrates considerations and decisions in scale design as the validation process from the start, which is a reminder that assessment is not just a quantitative exercise but a multifaceted process. • The design of a rating scale for first-year undergraduate academic writing is detailed. • Emphasis is placed on the role of reading in integrated writing scales. • Academic argumentation, rather than solely source-use mechanics, is considered. • Implications for construct operationalisation in academic evaluations are offered.

    doi:10.1016/j.asw.2025.100918

July 2024

  1. Examining the direct and indirect impacts of verbatim source use on linguistic complexity in integrated argumentative writing assessment
    Abstract

    Verbatim source use (VSU) in integrated argumentative writing tasks may enhance linguistic complexity of writing performance. This assistance might present an unequal advantage for test-takers across levels of writing proficiency, engendering validity and fairness concerns. While previous research has mostly examined the relationships between source use characteristics and proficiency levels, the relationship between VSU and linguistic complexity remains underexplored. To further unpack these relationships, this study examined both the direct impact of VSU on linguistic complexity of writing performances and its indirect impact through interaction with writing proficiency. Using natural language processing tools and techniques, we examined 34 linguistic complexity features and three VSU features of 3250 argumentative writing performances on a university-level English Placement Test (EPT). We performed exploratory factor analysis to identify linguistic complexity dimensions and applied mixed-effect models to examine how VSU features and proficiency level impacted these dimensions. Post-hoc analyses suggested weak direct impacts of different VSU features on linguistic complexity, which might reflect different essay writing strategies. However, no meaningful indirect impact was found. The findings help unravel the impact of VSU on argumentative writing and provide empirical evidence for validity arguments for integrated writing assessments.

    doi:10.1016/j.asw.2024.100868
  2. Does “more complexity” equal “better writing”? Investigating the relationship between form-based complexity and meaning-based complexity in high school EFL learners’ argumentative writing
    doi:10.1016/j.asw.2024.100867
  3. A large-scale corpus for assessing written argumentation: PERSUADE 2.0
    Abstract

    This research methods article introduces the open source PERSUADE 2.0 corpus. The PERSUADE 2.0 corpus comprises over 25,000 argumentative essays produced by 6th-12th grade students in the United States for 15 prompts on two writing tasks: independent and source-based writing. The PERSUADE 2.0 corpus also provides detailed individual and demographic information for each writer. The goal of the PERSUADE 2.0 corpus is to advance research into relationships between discourse elements, their effectiveness, writing quality, writing tasks and prompts, and demographic and individual differences.

    doi:10.1016/j.asw.2024.100865
  4. EFL students' syntactic complexity development in argumentative writing:A latent class growth analysis (LCGA) approach
    Abstract

    The study explored EFL students' development of syntactic complexity by employing the Latent Class Growth Analysis (LCGA) approach. A total of 214 tertiary EFL students from Southwest China were invited to write four argumentative essays over an academic semester. The unconditional models of LCGA were utilized to explore the optimal latent classes of students' development trajectories of syntactic complexity. The conditional models of LCGA were employed to investigate the predictive effect of English proficiency on the optimal latent classes. Results of the unconditional models revealed different latent classes of development trajectories for six indices of syntactic complexity rather than the remaining ones, which offers tentative evidence for the heterogeneity of L2 development trajectories. Results of the conditional models showed that English proficiency did not predict the membership in these latent classes. These results are discussed and implications for L2 instruction are attempted.

    doi:10.1016/j.asw.2024.100877

April 2024

  1. A comparison between input modalities and languages in source-based multilingual argumentative writing
    doi:10.1016/j.asw.2024.100813

July 2023

  1. Collaborating with ChatGPT in argumentative writing classrooms
    doi:10.1016/j.asw.2023.100752

April 2023

  1. Predicting Chinese EFL Learners’ Human‐rated Writing Quality in Argumentative Writing Through Multidimensional Computational Indices of Lexical Complexity
    doi:10.1016/j.asw.2023.100722

January 2023

  1. A multidimensional approach to assessing the effects of task complexity on L2 students’ argumentative writing
    doi:10.1016/j.asw.2022.100690
  2. Visual thinking and argumentative writing: A social-cognitive pairing for student writing development
    doi:10.1016/j.asw.2023.100694

October 2022

  1. The persuasive essays for rating, selecting, and understanding argumentative and discourse elements (PERSUADE) corpus 1.0
    Abstract

    This paper introduces the Persuasive Essays for Rating, Selecting, and Understanding Argumentative and Discourse Elements (PERSUADE) corpus.The PERSUADE corpus is large-scale corpus of writing with annotated discourse elements. The goal of the corpus is to spur the development of new, open-source scoring algorithms that identify discourse elements in argumentative writing to open new avenues for the development of automatic writing evaluation systems that focus more specifically on the semantic and organizational elements of student writing.

    doi:10.1016/j.asw.2022.100667
  2. Constructs of argumentative writing in assessment tools
    doi:10.1016/j.asw.2022.100675
  3. Using chatbots to scaffold EFL students’ argumentative writing
    doi:10.1016/j.asw.2022.100666

July 2022

  1. Diversity of Advanced Sentence Structures (DASS) in writing predicts argumentative writing quality and receptive academic language skills of fifth-to-eighth grade students
    doi:10.1016/j.asw.2022.100649

April 2022

  1. Dynamic assessment of argumentative writing: Mediating task response
    doi:10.1016/j.asw.2022.100606

October 2021

  1. The assessment of metadiscourse devices in English as a foreign language
    Abstract

    The objectives of this paper are to identify the metadiscourse devices used by English learners at the different levels of language acquisition established by the Common European Framework of Reference and to categorise them to facilitate the assessment and learning of textual and interpersonal devices. First, a learner corpus of essays written by English learners was compiled. Then, the metadiscourse devices were classified in different levels and categories. The results showed the lists and frequencies of metadiscourse devices. The examples aim to make additional and explicit connections between levels of language proficiency and assessment of metadiscourse devices. It can be stated, as a conclusion, that metadiscourse devices portray specific ways of argumentation in essay writing in different levels of EFL proficiency.

    doi:10.1016/j.asw.2021.100560

January 2021

  1. Syntactic complexity in L2 learners’ argumentative writing: Developmental stages and the within-genre topic effect
    doi:10.1016/j.asw.2020.100506

October 2018

  1. Connecting writing assessment with critical thinking: An exploratory study of alternative rhetorical functions and objects of enquiry in writing prompts
    doi:10.1016/j.asw.2018.09.001

April 2017

  1. Textual voice elements and voice strength in EFL argumentative writing
    doi:10.1016/j.asw.2017.02.002

October 2012

  1. The Critical Thinking Analytic Rubric (CTAR): Investigating intra-rater and inter-rater reliability of a scoring mechanism for critical thinking performance assessments
    doi:10.1016/j.asw.2012.07.002

January 2007

  1. Patterns of student writing in a critical thinking course: A quantitative analysis
    doi:10.1016/j.asw.2008.02.001

January 2004

  1. Assessing and teaching what we value: The relationship between college-level writing and critical thinking abilities
    doi:10.1016/j.asw.2004.01.003

October 2002

  1. Developing a performance-based assessment of students’ critical thinking skills
    doi:10.1016/s1075-2935(02)00031-4

January 1998

  1. Commentary on “validation of a scheme for assessing argumentative writing of middle school students”
    doi:10.1016/s1075-2935(99)80010-5
  2. Validation of a scheme for assessing argumentative writing of middle school students
    doi:10.1016/s1075-2935(99)80009-9