Using AI to understand students’ self-assessments of their writing
Abstract
This study focuses on a generative AI approach to facilitate qualitative analysis in Writing Studies research. We gathered 13,336 one-sentence to one-paragraph responses written by 3,334 incoming students in a directed self-placement program administered at a large R1 U.S. university. In these responses, students describe their high school writing experience and college writing expectations. In stage one of the project, we pilot the use of Retrieval-Augmented Generation to expedite the selection of relevant responses for a topic—in this case, students’ positive self-assessments as writers. The selected responses were then compared to a random sample and rated by three faculty with writing expertise. In stage two, these faculty generated codes and themes from a subset of the responses, incorporating ChatGPT-4 through the stages of thematic analysis. Results show that the use of AI expedites and enhances qualitative analysis, but human participation in the process is still essential. We suggest a machine-in-the-loop framework with which Writing Studies researchers can more readily integrate generative AI to study large corpora of student writing.
- Journal
- Journal of Writing Research
- Published
- 2026-02-17
- DOI
- 10.17239/jowr-2026.17.03.07
- CompPile
- Open Access
- OA PDF Diamond
- Topics
- Export
- BibTeX RIS
Citation Context
Cited by in this index (0)
No articles in this index cite this work.
References (0)
No references on file for this article.
Related Articles
-
The Peer Review Sep 2025Ana Raquel Fialho Ferreira Campos; João Tiago Gaspar Cozechen; Elaine Pereira Lustosa; Marcos Angel De Carvalho Eing; Leonardo Schimiloski
-
Business and Professional Communication Quarterly May 2025Emily Barrow DeJeu; Beth A. Walter; Michael C. Melville
-
Journal of Academic Writing Apr 2025Gea Dreschler; Abby Gambrel; Jens Branum
-
Journal of Academic Writing Feb 2025Thomas Lancaster
-
Business and Professional Communication Quarterly May 2024Daneshwar Sharma; Himanshi Pandey