Using AI to understand students’ self-assessments of their writing
Abstract
This study focuses on a generative AI approach to facilitate qualitative analysis in Writing Studies research. We gathered 13,336 one-sentence to one-paragraph responses written by 3,334 incoming students in a directed self-placement program administered at a large R1 U.S. university. In these responses, students describe their high school writing experience and college writing expectations. In stage one of the project, we pilot the use of Retrieval-Augmented Generation to expedite the selection of relevant responses for a topic—in this case, students’ positive self-assessments as writers. The selected responses were then compared to a random sample and rated by three faculty with writing expertise. In stage two, these faculty generated codes and themes from a subset of the responses, incorporating ChatGPT-4 through the stages of thematic analysis. Results show that the use of AI expedites and enhances qualitative analysis, but human participation in the process is still essential. We suggest a machine-in-the-loop framework with which Writing Studies researchers can more readily integrate generative AI to study large corpora of student writing.
- Journal
- Journal of Writing Research
- Published
- 2026-02-17
- DOI
- 10.17239/jowr-2026.17.03.07
- CompPile
- Search in CompPile ↗
- Open Access
- OA PDF Diamond
- Topics
- Export
- BibTeX RIS
Citation Context
Cited by in this index (0)
No articles in this index cite this work.
Cites in this index (0)
No references match articles in this index.
Related Articles
-
The Peer Review Sep 2025Ana Raquel Fialho Ferreira Campos; João Tiago Gaspar Cozechen; Elaine Pereira Lustosa; Marcos Angel De Carvalho Eing; Leonardo Schimiloski
-
Business and Professional Communication Quarterly May 2025Emily Barrow DeJeu; Beth A. Walter; Michael C. Melville
-
Journal of Academic Writing Apr 2025Gea Dreschler; Abby Gambrel; Jens Branum
-
Journal of Academic Writing Feb 2025Thomas Lancaster
-
Business and Professional Communication Quarterly May 2024Daneshwar Sharma; Himanshi Pandey