Ben Markey
2 articles-
Presenting and Making Relevant: Analyzing Teaching Assistant Perceptions of Writing in Statistics Using Semantic Frames ↗
Abstract
Background: Instructors in STEM fields help prepare students to be effective communicators in the workplace, partially through instruction of professional genres such as client-facing reports. At the same time, class sizes are increasing, and writing assessment often falls to teaching assistants (TAs). Literature review: Research suggests that TAs possess a maturing but inchoate sense of writing in their field, which potentially complicates their ability to deliver quality feedback. This study uses frame semantics, a form of discourse analysis, to probe TAs for their beliefs about writing in statistics. Research questions: 1. When asked to describe the function and role of writing in statistics, what lexical verbs do TA informants use? 2. What frames are invoked by those verbs? 3. How do the invoked semantic frames position writing in relation to disciplinary and professional work in the field? Research methodology: This study interviewed three TAs from an introductory statistics course about their perceptions of writing in statistics. Frame semantics was used to analyze TA responses. Results: Less experienced TAs tended to perceive writing as a means of presentation, which entailed a weak sense of the role of rhetoric in technical communication and a muddied understanding of writing assessment. The more advanced TA perceived writing as a means of contextualizing statistical evidence for particular audiences. Conclusion: Due to their maturing perceptions of writing in their disciplines, TAs might not possess the ability to deliver quality formative feedback. One means of support for these TAs may be opportunities to discuss assessment decisions with one another, thereby calibrating against available expectations and rubrics.
-
Abstract
ChatGPT and other LLMs are at the forefront of pedagogical considerations in classrooms across the academy. Many studies have spoken to the technology’s capacity to generate one-off texts in a variety of genres. This study complements those by inquiring into its capacity to generate compelling texts at scale. In this study, we quantitatively and qualitatively analyze a small corpus of generated texts in two genres and gauge it against novice and published academic writers along known dimensions of linguistic variation. Theoretically, we position and historicize ChatGPT as a writing technology and consider the ways in which generated text may not be congruent with established trajectories of writing development in higher education. Our study found that generated texts are more informationally dense than authored texts and often read as dialogically closed, “empty,” and “fluffy.” We close with a discussion of potentially explanatory linguistic features, as well as relevant pedagogical implications.