Computers and Composition Dec 2024 Open Access

When generative artificial intelligence meets multimodal composition: Rethinking the composition process through an AI-assisted design project

Jialei Jiang University of Pittsburgh

Abstract

• This study explores GenAI's role in multimodal composition, including Adobe Firefly and DALL·E. • GenAI reshapes the composition stages of invention, designing, and revising. • Despite its limitations, GenAI offers alternative solutions to wicked problems. • Post-GenAI use, students critically revise and iterate their compositions. • The study contributes to future research and teaching of AI-assisted composition. This study explores the integration of generative artificial intelligence (GenAI) design technologies, including Adobe Firefly and DALL·E, into the teaching and learning of multimodal composition. Through focus group discussions and case studies, this paper demonstrates the potential of GenAI in reshaping the various stages of the composition process, including invention, designing, and revising. The findings reveal that GenAI technologies have the potential to enhance students’ multimodal composition practices and offer alternative solutions to the wicked problems encountered during the design process. Specifically, GenAI facilitates invention by offering design inspirations and enriches designing by expanding, removing, and editing the student-produced design contents. The students in this study also shared their critical stance on the revision process by modifying and iterating their designs after their uses of GenAI. Through showcasing both the opportunities and challenges of GenAI technologies, this paper contributes to the ongoing scholarly conversations on multimodal composition and pedagogy. Moreover, the paper offers implications for the future research and teaching of GenAI-assisted multimodal composition projects, with the aim of encouraging thoughtful integration of GenAI technologies to foster critical AI literacy among college composition students.

Journal
Computers and Composition
Published
2024-12-01
DOI
10.1016/j.compcom.2024.102883
Open Access
OA PDF Hybrid
Topics
multimodality artificial intelligence multilingual writers

Citation Context

Cited by in this index (6)

  1. Jiang et al. (2026)
    Designing experience: Multimodal UX in the expanding field of composition
    Computers and Composition
  2. Quigley et al. (2026)
    Kiosk! An interactive touchscreen project for multimodal UX composition learners
    Computers and Composition
  3. Verhulsdonck et al. (2026)
    Expanding Human-in-the-Loop: Critical Sensemaking for Technical and Professiona…
    Journal of Technical Writing and Communication
  4. Jiang et al. (2025)
    Augmenting User Experience Design with Multimodal Generative Artificial Intelli…
    Technical Communication Quarterly
  5. Ghimire (2025)
    Utilizing ChatGPT to integrate world English and diverse knowledge: A transnati…
    Computers and Composition
Show all 6 →
  1. Jiang et al. (2024)
    From Hype to Practice: Reinterpreting the Writing Process Through Technical Wri…
    Technical Communication Quarterly

Cites in this index (14)

  1. Tham et al. (2024)
    Understanding Writing Instructors’ Feelings toward the Affordances of Multimoda…
    College Composition and Communication
  2. Bedington et al. (2024)
    Writing with generative AI and human-machine teaming: Insights and recommendati…
    Computers and Composition
  3. Cummings et al. (2024)
    Generative AI in first-year writing: An early analysis of affordances, limitati…
    Computers and Composition
  4. Knowles (2024)
    Machine-in-the-loop writing: Optimizing the rhetorical load
    Computers and Composition
  5. Li (2024)
    A “Dance of storytelling”: Dissonances between substance and style in collabora…
    Computers and Composition
Show all 14 →
  1. Pigg (2024)
    Research writing with ChatGPT: A descriptive embodied practice framework
    Computers and Composition
  2. Vetter et al. (2024)
    Towards a framework for local interrogation of AI ethics: A case study on text …
    Computers and Composition
  3. Jiang (2024)
    Composing to Enact Affective Agency: Engaging Multimodal Antiracist Pedagogy in…
    College Composition and Communication
  4. Tham et al. (2022)
    Extending Design Thinking, Content Strategy, and Artificial Intelligence into T…
    Journal of Technical Writing and Communication
  5. Leverenz (2014)
    Design Thinking and the Wicked Problem of Teaching Writing
    Computers and Composition
  6. Purdy (2014)
    What Can Design Thinking Offer Writing Studies?
    College Composition and Communication
  7. Wickman (2014)
    Wicked Problems in Technical Communication
    Journal of Technical Writing and Communication
  8. Marback (2009)
    Embracing Wicked Problems: The Turn to Design in Composition Studies
    College Composition and Communication
  9. FLOWER et al. (1984)
    Images, Plans, and Prose
    Written Communication
Also cites 11 works outside this index ↓
  1. Daniel (2023)
    Cultivating writerly virtues: Critical human elements of multimodal writing in the age of…
    Journal of Adolescent & Adult Literacy  ↗
  2. Deng (2023)
    A meta-analysis and systematic review of the effect of chatbot technology use in sustaina…
    Sustainability  ↗
  3. Hafner (2020)
    Assessing digital multimodal composing in second language writing: Towards a process-base…
    Journal of Second Language Writing  ↗
  4. Jiang (2024)
    Emotions are what will draw people in”: A study of critical affective literacy through di…
    Journal of Adolescent & Adult Literacy  ↗
  5. Kang (2023)
    Beyond ChatGPT: Multimodal generative AI for L2 writers
    Journal of Second Language Writing  ↗
  6. Leander (2020)
    Critical literacy for a posthuman world: When people read, and become, with machines
    British Journal of Educational Technology  ↗
  7. Liu (2024)
    Investigating students’ cognitive processes in generative AI-assisted digital multimodal …
    Computers & Education  ↗
  8. Malik (2023)
    Exploring artificial intelligence in academic essay: Higher education student's perspective
    International Journal of Educational Research Open  ↗
  9. Rittel (1973)
    Dilemmas in a general theory of planning
    Policy Sciences  ↗
  10. Yan (2023)
    Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation
    Education and Information Technologies  ↗
  11. Zhang (2024)
    ChatGPT: Enhancing learner agency and multilingual practices in English writing class
    Journal of Education, Humanities and Social Sciences  ↗
CrossRef global citation count: 27 View in citation network →

Related Articles

  1. Written Communication Oct 2025
    Ownership, Accuracy, and Aesthetics: University Writers’ Perceptions of GenAI Poetry ↗
    Justin Nicholes
    multimodality artificial intelligence multilingual writers creative writing
  2. Journal of Technical Writing and Communication Apr 2025
    Synthetic Genres: Expert Genres, Non-Specialist Audiences, and Misinformation in the Artificial Intelligence Age ↗
    Brad Mehlenbacher; Ana Patricia Balbon; Ashley Rose Mehlenbacher
    genre theory multimodality artificial intelligence scientific writing multilingual writers
  3. Technical Communication Quarterly Mar 2025
    Augmenting User Experience Design with Multimodal Generative Artificial Intelligence: A Study of Technical Communication Students ↗
    Jialei Jiang; Gustav Verhulsdonck
    multimodality artificial intelligence technical communication multilingual writers
  4. Computers and Composition Mar 2025
    Editorial for special issue: Digital multimodal composing in the era of artificial intelligence ↗
    Fei Victor Lim; Øystein Gilje; Emilia Djonov
    multimodality artificial intelligence multilingual writers
  5. Journal of Business and Technical Communication Jan 2025
    Rhetorics of Authenticity: Ethics, Ethos, and Artificial Intelligence ↗
    Adrianna Deptula; Paul Thompson Hunter; Richard Johnson-Sheehan
    multimodality artificial intelligence professional writing multilingual writers voice and style argument