ChatGPT and other LLMs are at the forefront of pedagogical considerations in classrooms across the academy. Many studies have spoken to the technology’s capacity to generate one-off texts in a variety of genres. This study complements those by inquiring into its capacity to generate compelling texts at scale. In this study, we quantitatively and qualitatively analyze a small corpus of generated texts in two genres and gauge it against novice and published academic writers along known dimensions of linguistic variation. Theoretically, we position and historicize ChatGPT as a writing technology and consider the ways in which generated text may not be congruent with established trajectories of writing development in higher education. Our study found that generated texts are more informationally dense than authored texts and often read as dialogically closed, “empty,” and “fluffy.” We close with a discussion of potentially explanatory linguistic features, as well as relevant pedagogical implications.

Recently, formative feedback in writing instruction has been supported by technologies generally referred to as Automated Writing Evaluation tools. However, such tools are limited in their capacity to explore specific disciplinary genres, and they have shown mixed results in student writing improvement. We explore how technology-enhanced writing interventions can positively affect student attitudes toward and beliefs about writing, both reinforcing content knowledge and increasing student motivation. Using a student-facing text-visualization tool called Write & Audit, we hosted revision workshops for students (n = 30) in an introductory-level statistics course at a large North American University. The tool is designed to be flexible: instructors of various courses can create expectations and predefine topics that are genre-specific. In this way, students are offered non-evaluative formative feedback which redirects them to field-specific strategies. To gauge the usefulness of Write & Audit, we used a previously validated survey instrument designed to measure the construct model of student motivation (Ling et al. 2021). Our results show significant increases in student self-efficacy and beliefs about the importance of content in successful writing. We contextualize these findings with data from three student think-aloud interviews, which demonstrate metacognitive awareness while using the tool. Ultimately, this exploratory study is non-experimental, but it contributes a novel approach to automated formative feedback and confirms the promising potential of Write & Audit.

Text-driven, quantitative methods provide new ways to analyze student writing, by uncovering recurring grammatical features and related stylistic effects that remain tacit to students and those who read and evaluate student writing. To date, however, these methods are rarely used in research on students transitioning into US postsecondary writing, and especially rare are studies of student writing that is already scored according to high-stakes writing expectations. This study offers a corpus-based, comparative analysis of higher- and lower-scoring Advanced Placement (AP) exams in English, revealing statistically significant syntactic patterns that distinguish higher-scoring exams according to “informational production” and lower-scoring essays according to “involved” or “interactional” production (Biber, 1988). These differences contribute to what we label emphatic generality in the lower-scoring essays, in which writers tend to foreground human actors, including themselves. In contrast, patterns in higher-scoring essays achieve what we call elaborated specificity, by focusing on and explicating specific, often abstract, concepts.These findings help uncover what is rewarded (or not) in high-stakes writing assessments and show that some students struggle with register awareness. A related implication, then, is the importance of teaching register awareness to students at the late secondary and early university level—students who are still relative novices, but are being invited to compose informationally dense prose. Such register considerations, and specific features revealed in this study, provide ways to help demystify privileged writing forms for students, particularly students for whom academic writing may seem distant from their own communicative practices and ambitions.