ChatGPT and other LLMs are at the forefront of pedagogical considerations in classrooms across the academy. Many studies have spoken to the technology’s capacity to generate one-off texts in a variety of genres. This study complements those by inquiring into its capacity to generate compelling texts at scale. In this study, we quantitatively and qualitatively analyze a small corpus of generated texts in two genres and gauge it against novice and published academic writers along known dimensions of linguistic variation. Theoretically, we position and historicize ChatGPT as a writing technology and consider the ways in which generated text may not be congruent with established trajectories of writing development in higher education. Our study found that generated texts are more informationally dense than authored texts and often read as dialogically closed, “empty,” and “fluffy.” We close with a discussion of potentially explanatory linguistic features, as well as relevant pedagogical implications.

Text-driven, quantitative methods provide new ways to analyze student writing, by uncovering recurring grammatical features and related stylistic effects that remain tacit to students and those who read and evaluate student writing. To date, however, these methods are rarely used in research on students transitioning into US postsecondary writing, and especially rare are studies of student writing that is already scored according to high-stakes writing expectations. This study offers a corpus-based, comparative analysis of higher- and lower-scoring Advanced Placement (AP) exams in English, revealing statistically significant syntactic patterns that distinguish higher-scoring exams according to “informational production” and lower-scoring essays according to “involved” or “interactional” production (Biber, 1988). These differences contribute to what we label emphatic generality in the lower-scoring essays, in which writers tend to foreground human actors, including themselves. In contrast, patterns in higher-scoring essays achieve what we call elaborated specificity, by focusing on and explicating specific, often abstract, concepts.These findings help uncover what is rewarded (or not) in high-stakes writing assessments and show that some students struggle with register awareness. A related implication, then, is the importance of teaching register awareness to students at the late secondary and early university level—students who are still relative novices, but are being invited to compose informationally dense prose. Such register considerations, and specific features revealed in this study, provide ways to help demystify privileged writing forms for students, particularly students for whom academic writing may seem distant from their own communicative practices and ambitions.