Assessing Writing

1017 articles
Year: Topic:
Export:

April 2024

  1. Establishing analytic score profiles for large-scale L2 writing assessment: The case of the CET-4 writing test
    doi:10.1016/j.asw.2024.100826
  2. Editorial Board
    doi:10.1016/s1075-2935(24)00046-1
  3. Visualizing formative feedback in statistics writing: An exploratory study of student motivation using DocuScope Write & Audit
    Abstract

    Recently, formative feedback in writing instruction has been supported by technologies generally referred to as Automated Writing Evaluation tools. However, such tools are limited in their capacity to explore specific disciplinary genres, and they have shown mixed results in student writing improvement. We explore how technology-enhanced writing interventions can positively affect student attitudes toward and beliefs about writing, both reinforcing content knowledge and increasing student motivation. Using a student-facing text-visualization tool called Write & Audit, we hosted revision workshops for students (n = 30) in an introductory-level statistics course at a large North American University. The tool is designed to be flexible: instructors of various courses can create expectations and predefine topics that are genre-specific. In this way, students are offered non-evaluative formative feedback which redirects them to field-specific strategies. To gauge the usefulness of Write & Audit, we used a previously validated survey instrument designed to measure the construct model of student motivation (Ling et al. 2021). Our results show significant increases in student self-efficacy and beliefs about the importance of content in successful writing. We contextualize these findings with data from three student think-aloud interviews, which demonstrate metacognitive awareness while using the tool. Ultimately, this exploratory study is non-experimental, but it contributes a novel approach to automated formative feedback and confirms the promising potential of Write & Audit.

    doi:10.1016/j.asw.2024.100830
  4. The associations among growth mindsets, the ideal L2 writing self, and L2 writing enjoyment and their impacts on L2 English writing performance: A gender difference perspective
    doi:10.1016/j.asw.2024.100832
  5. Book review
    doi:10.1016/j.asw.2024.100828
  6. Effects of task-based language teaching on functional adequacy in L2 writing
    doi:10.1016/j.asw.2024.100838
  7. Exploring the dynamics of student engagement with receiving peer feedback in L2 writing
    doi:10.1016/j.asw.2024.100842
  8. Book review
    doi:10.1016/j.asw.2024.100835
  9. Is the variation in syntactic complexity features observed in argumentative essays produced by B1 level EFL learners in Finland and Pakistan attributable exclusively to their L1?
    Abstract

    This study has explored the syntactic complexity features of English learners at the B1 Common European Framework of Reference (CEFR) (CoE, 2001) level from both Pakistan and Finland. The learners in question were taught English as a Foreign Language (EFL) using different pedagogical methods. This study took into account various factors including the learners' proficiency level, age, and grade, as well as variations in their native language. To assess the impact of the learners' native language and pedagogical methods on syntactic complexity features, twelfth grade EFL students from Upper-Secondary schools in both nations were given identical instructions and time limits to complete an English academic essay on the same topic. The study utilized L2 syntactic complexity analyzer (L2SCA) to extract fourteen syntactic complexity features, and Mann-Whitney U Tests were used to analyze the differences in the syntactic complexity features between the two groups. The study has revealed significant differences between Finnish and Pakistani EFL learners due to variations in their native language and the effects of pedagogical methods on syntactic complexity features. The implications of this study extend to language testing and assessment, the CEFR framework, and pedagogy in both Finland and Pakistan.

    doi:10.1016/j.asw.2024.100839
  10. Assessing writing and spelling interest and self-beliefs: Does the type of pictorial support affect first and third graders’ responses?
    Abstract

    An array of pictorial supports (e.g., emojis, geometrical figures, animals) is often used in studies assessing young students’ writing motivation with Likert scales. However, although these images may influence the students’ responses, sufficient rationales for these choices are often absent from the studies. To the best of our knowledge, the present study is the first to investigate two different types of pictorial support (circles vs. faces) in Likert scales assessing first and third graders’ writing interest, self-concept, and spelling interest and self-efficacy. The samples consist of 2197 first graders (mean age 6.8 years) and 1740 third graders (mean age 8.4 years). Results show statistically significant differences among the scales indicating that when face-scales are used, first-graders skip motivation items more often, and students in both grades avoid the minimum values of the scale more often. Gender differences are also found indicating that when face-scales are used, boys in third grade avoid maximum values more often, and girls in both grades avoid the minimum values more often. These findings suggest that the use of circle-scales compared to face-scales seem more appropriate in scales measuring young students’ writing and spelling interest and self-beliefs.

    doi:10.1016/j.asw.2024.100833
  11. Validity of automated essay scores for elementary-age English language learners: Evidence of bias?
    doi:10.1016/j.asw.2024.100815
  12. Assessing video game narratives: Implications for the assessment of multimodal literacy in ESP
    Abstract

    Research into the contribution of multimodality to language learning is gaining momentum. While most studies pave the way for new understandings of language teaching and learning, there is an increasing demand for comprehensive assessment practices, particularly within higher education contexts. A few studies have emphasized the importance of reflecting on and establishing criteria for the assessment of multimodal literacy. This is necessary to understand students’ contributions in detail and to provide them with effective support in developing their multimodal skills. This study discusses the assessment of multimodal writing in English for Specific Purposes (ESP) contexts. It presents the design of an analytical tool for assessing multimodal texts and provided an example of its application. This tool covers assessment categories such as language use, content expression, interpersonal meaning, multimodality, and creativity and originality. As an example, we focus on the multimodal writing of a video game narrative, a genre that requires the integration of multiple modes of communication to convey meaning more effectively. Finally, this study offers pedagogical insights into the assessment of multimodal literacy in ESP.

    doi:10.1016/j.asw.2024.100809
  13. The assessment of writing in languages other than English (LOTE)
    doi:10.1016/j.asw.2024.100840
  14. A comparison between input modalities and languages in source-based multilingual argumentative writing
    doi:10.1016/j.asw.2024.100813
  15. Characteristics of students’ task representation and its association with argumentative integrated writing performance
    Abstract

    Task representation denotes students’ interpretation in which what a learning or assessment task required them to do. An argumentative integrated writing task which involves the use of reading materials as claims or evidences for composing an essay, makes the role of task representation more critical than others, as writers may be confused with whether their task is to focus on synthesizing the reading materials that they comprehend, or expressing their own views. With the aim of exploring the characteristics of task representation and its association with integrated writing, this study invited 474 secondary four students from Hong Kong to participate in think aloud writing protocol followed by stimulated recall interview (36 participants), and complete an integrated writing task and a questionnaire (438 participants). Three factors of the task representation were identified as source use, rhetorical purpose and text format, and significant positive correlations were found between the three factors and integrated writing performance. Theoretical and pedagogical implications are discussed.

    doi:10.1016/j.asw.2024.100845
  16. Volume 60 editorial
    doi:10.1016/j.asw.2024.100857
  17. Engagement with supervisory feedback on master’s theses: Do supervisors and students see eye to eye?
    doi:10.1016/j.asw.2024.100841
  18. Is CJ a valid, reliable form of L2 writing assessment when texts are long, homogeneous in proficiency, and feature heterogeneous prompts?
    doi:10.1016/j.asw.2024.100843
  19. Writing assessment and feedback literacy: Where do we stand and where can we go?
    doi:10.1016/j.asw.2024.100829
  20. Book review
    doi:10.1016/j.asw.2024.100831

January 2024

  1. Editorial
    doi:10.1016/j.asw.2024.100816
  2. Reading, receiving, revising: A case study on the relationship between peer review and revision in writing-to-learn
    doi:10.1016/j.asw.2024.100808
  3. Do I need feedback or avoid it in L2 writing? Impacts of self-efficacy and shyness on feedback-seeking behavior
    doi:10.1016/j.asw.2023.100805
  4. Noun phrase complexity and second language Chinese proficiency: An analysis of narratives by Korean learners of Chinese
    doi:10.1016/j.asw.2024.100810
  5. Book review
    doi:10.1016/j.asw.2024.100814
  6. Reading the reader through raciolinguistic ideologies: An investigation of the evidence students present in self- placement
    doi:10.1016/j.asw.2023.100792
  7. Amplifying test-taker voices in the validation of L2 writing assessment tasks
    doi:10.1016/j.asw.2023.100790
  8. A mixed Rasch model analysis of multiple profiles in L2 writing
    Abstract

    The present study used the Mixed Rasch Model (MRM) to identify multiple profiles in L2 students’ writing with regard to several linguistic features, including content, organization, grammar, vocabulary, and mechanics. To this end, a pool of 500 essays written by English as a foreign language (EFL) students were rated by four experienced EFL teachers using the Empirically-derived Descriptor-based Diagnostic (EDD) checklist. The ratings were subjected to MRM analysis. Two distinct profiles of L2 writers emerged from the sample analyzed including: (a) Sentence-Oriented and (b) Paragraph-Oriented L2 Writers. Sentence-Oriented L2 Writers tend to focus more on linguistic features, such as grammar, vocabulary, and mechanics, at the sentence level and try to utilize these subskills to generate a written text. However, Paragraph-Oriented Writers are inclined to move beyond the boundaries of a sentence and attend to the structure of a whole paragraph using higher-order features such as content and organization subskills. The two profiles were further examined to capture their unique features. Finally, the theoretical and pedagogical implications of the identification of L2 writing profiles and suggestions for further research are discussed.

    doi:10.1016/j.asw.2023.100803
  9. The motivational aspect of feedback: A meta-analysis on the effect of different feedback practices on L2 learners’ writing motivation
    doi:10.1016/j.asw.2023.100802
  10. Unlocking writing success: Building assessment literacy for students and teachers through effective interventions
    doi:10.1016/j.asw.2023.100804
  11. Editorial Board
    doi:10.1016/s1075-2935(24)00014-x
  12. Book review
    doi:10.1016/j.asw.2023.100807
  13. Assessing metacognition-based student feedback literacy for academic writing
    doi:10.1016/j.asw.2024.100811
  14. Effects of errors on ratings of writing performances – Evidence from a high-stakes exam
    Abstract

    The study reported in the paper starts with a hypothesis that errors observable in writing performances can account for much of the variability of the ratings awarded to them. The assertion is that this may be the case even when prescribed rating criteria explicitly direct rater focus towards successfully performed aspects of a writing performance rather than towards errors. The hypothesis is tested on a sample of texts rated independently of the study, using a five-point analytic rating scale involving ‘Can do’-like descriptors. The correlation between errors and ratings is ascertained using ordinal logistic regression, with Pseudo R2 of 0.51 discerned overall. Thus, with roughly 50% of score variability explainable by error occurrences, the stated hypothesis is considered confirmed. The study goes on to discuss the consequences of the findings and their potential employ in assessment of writing beyond the local assessment context.

    doi:10.1016/j.asw.2023.100806

October 2023

  1. Profiling support in literacy development: Use of natural language processing to identify learning needs in higher education
    doi:10.1016/j.asw.2023.100787
  2. Editorial
    doi:10.1016/j.asw.2023.100793
  3. Assessing Korean writing ability through a scenario-based assessment approach
    doi:10.1016/j.asw.2023.100766
  4. Student engagement with peer feedback in L2 writing: Insights from reflective journaling and revising practices
    doi:10.1016/j.asw.2023.100784
  5. Growth mindset and emotions in tandem: Their effects on L2 writing performance based on writers’ proficiency levels
    doi:10.1016/j.asw.2023.100785
  6. Feedback literacy in writing research and teaching: Advancing L2 WCF research agendas
    Abstract

    Research on corrective feedback (CF) has developed from its original focus on identifying which type of CF is most effective for developing L2 language learners’ grammatical accuracy to focusing on how learners use CF. Underpinning this is the assumption that learners know what to do with CF when they receive it. The concept of “feedback literacy” challenges this assumption. Carless and Boud (2018), define feedback literacy as “the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies” (p. 1316). Our intention in this paper is to reflect on the manner in which theoretical and empirical work on feedback literacy can contribute to advancing L2 written corrective feedback (WCF) research agendas. Central in our proposal is the partially under-researched aspect of experience in terms of the L2 writers’ educational background experience, particularly experience with L1 and L2 writing. We further argue that how learners were taught L1 writing and how the L1 educational culture/ society values writing can impact on how learners approach L2 writing tasks and accompanying feedback. Implications of this inclusive view of the learner for future research and pedagogy is discussed.

    doi:10.1016/j.asw.2023.100786
  7. Insights from lexical and syntactic analyses of a French for academic purposes assessment
    Abstract

    With the objective of improving writing assessment of language instruction, we examine the lexical and syntactic features in two corpora of high and low scoring French texts of the Test du Certificat de Compétence en Langue Seconde (Second Language Certification Test; TCCLS) at the University of Ottawa (uOttawa). We first situate the test in its local context, demonstrating how our research objectives are born from specific needs to improve student outcomes. We then describe our creation of two corpora of high and low performing test takers, followed by lexical bundle (LB) analyses (Phase 1) and further linguistic complexity analyses with a French-language tool (Phase 2). Results indicate that high level writers used more LBs and borrowed more text from the prompt than low level writers. In addition, specific elements of linguistic complexity were identified, suggesting high level writers produced texts that were lexically richer and more syntactically advanced. We discuss the importance of these findings in improving our writing instruction, as well as the challenges of adapting tools and approaches traditionally associated with English to French.

    doi:10.1016/j.asw.2023.100789
  8. Understanding EFL students’ feedback literacy development in academic writing: A longitudinal case study
    doi:10.1016/j.asw.2023.100770
  9. The effects of task complexity and language aptitude on EFL learners’ writing performance
    doi:10.1016/j.asw.2023.100791
  10. Automated analysis of cohesive features in L2 writing: Examining effects of task complexity and task repetition
    doi:10.1016/j.asw.2023.100783
  11. Developments in learners’ affective engagement with written peer feedback: The affordances of in situ translanguaging
    doi:10.1016/j.asw.2023.100788
  12. Editorial Board
    doi:10.1016/s1075-2935(23)00105-8
  13. Book review
    doi:10.1016/j.asw.2023.100782

July 2023

  1. Peer-feedback of an occluded genre in the Spanish language classroom: A case study
    Abstract

    Learning how to write occluded genres is an elusive task (Swales, 1996) – even more so in the case of students writing in a second or additional language. To achieve discourse competence in the use of one of these genres, in this case the ‘statement of purpose’ typical of post-graduate programme admission forms, it is first necessary to fully understand its features at both the macrotextual and microlinguistic levels (Gillaerts, 2003; Bhatia, 2004). This qualitative study focuses on the writing of learners of Spanish as an additional language to analyse whether feedback provided by peers impacts the quality of the statements of purpose they write. Through a dual discourse analysis of their written work and in-class interactions during peer- feedback sessions, our study finds that, when properly trained and using tailored assessment tools, students can use peer-assessment profitably to improve the quality of their statements of purpose, as well as to acquire appropriate metalanguage to guide others. Our results thus reconfirm the beneficial effects of helping students to achieve feedback literacy.

    doi:10.1016/j.asw.2023.100756
  2. Diagnosing Chinese college-level English as a Foreign Language (EFL) learners’ integrated writing capability: A Log-linear Cognitive Diagnostic Modeling (LCDM) study
    doi:10.1016/j.asw.2023.100730
  3. Editorial
    doi:10.1016/j.asw.2023.100772