Computers and Composition Dec 2024 Open Access

When generative artificial intelligence meets multimodal composition: Rethinking the composition process through an AI-assisted design project

Jialei Jiang University of Pittsburgh

Abstract

• This study explores GenAI's role in multimodal composition, including Adobe Firefly and DALL·E. • GenAI reshapes the composition stages of invention, designing, and revising. • Despite its limitations, GenAI offers alternative solutions to wicked problems. • Post-GenAI use, students critically revise and iterate their compositions. • The study contributes to future research and teaching of AI-assisted composition. This study explores the integration of generative artificial intelligence (GenAI) design technologies, including Adobe Firefly and DALL·E, into the teaching and learning of multimodal composition. Through focus group discussions and case studies, this paper demonstrates the potential of GenAI in reshaping the various stages of the composition process, including invention, designing, and revising. The findings reveal that GenAI technologies have the potential to enhance students’ multimodal composition practices and offer alternative solutions to the wicked problems encountered during the design process. Specifically, GenAI facilitates invention by offering design inspirations and enriches designing by expanding, removing, and editing the student-produced design contents. The students in this study also shared their critical stance on the revision process by modifying and iterating their designs after their uses of GenAI. Through showcasing both the opportunities and challenges of GenAI technologies, this paper contributes to the ongoing scholarly conversations on multimodal composition and pedagogy. Moreover, the paper offers implications for the future research and teaching of GenAI-assisted multimodal composition projects, with the aim of encouraging thoughtful integration of GenAI technologies to foster critical AI literacy among college composition students.

Journal
Computers and Composition
Published
2024-12-01
DOI
10.1016/j.compcom.2024.102883
CompPile
Search in CompPile ↗
Open Access
OA PDF Hybrid
Topics
writing pedagogy teacher development revision qualitative research multimodality artificial intelligence literacy studies
Export
BibTeX RIS

Citation Context

Cited by in this index (9)

  1. Anders et al. (2026)
    Threshold concepts for writing with AI: Experimentation, expertise, agency
    Computers and Composition
  2. Deets (2026)
    A quantitative, computational investigation of Computers and Composition: Using…
    Computers and Composition
  3. Jiang et al. (2026)
    User experience in composition: Rethinking the past and mapping the future of w…
    Computers and Composition
  4. Jiang et al. (2026)
    Designing experience: Multimodal UX in the expanding field of composition
    Computers and Composition
  5. Quigley et al. (2026)
    Kiosk! An interactive touchscreen project for multimodal UX composition learners
    Computers and Composition
Show all 9 →
  1. Verhulsdonck et al. (2026)
    Expanding Human-in-the-Loop: Critical Sensemaking for Technical and Professiona…
    Journal of Technical Writing and Communication
  2. Jiang et al. (2025)
    Augmenting User Experience Design with Multimodal Generative Artificial Intelli…
    Technical Communication Quarterly
  3. Ghimire (2025)
    Utilizing ChatGPT to integrate world English and diverse knowledge: A transnati…
    Computers and Composition
  4. Jiang et al. (2024)
    From Hype to Practice: Reinterpreting the Writing Process Through Technical Wri…
    Technical Communication Quarterly

References (50) · 14 in this index

  1. AlAfnan (2023)
    ChatGPT as an educational tool: Opportunities, challenges, and recommendations for commun…
    Journal of Artificial Intelligence and Technology
  2. Alexander (2014)
    On multimodality: New media in composition studies
  3. Ball (2022)
    Writer/designer: A guide to making multimodal projects
  4. Bedington et al. (2024)
    Writing with generative AI and human-machine teaming: Insights and recommendati…
    Computers and Composition
  5. Berthoff (1982)
    Forming, thinking, writing: The composing imagination
Show all 50 →
  1. (2013)
    Multimodal literacies and emerging genres
  2. Braidotti (2013)
    The posthuman
  3. Burriss (2024)
    Critical posthumanist literacy: Building theory for reading, writing, and living ethicall…
    Reading Research Quarterly
  4. Cummings et al. (2024)
    Generative AI in first-year writing: An early analysis of affordances, limitati…
    Computers and Composition
  5. Daniel (2023)
    Cultivating writerly virtues: Critical human elements of multimodal writing in the age of…
    Journal of Adolescent & Adult Literacy  ↗
  6. Deng (2023)
    A meta-analysis and systematic review of the effect of chatbot technology use in sustaina…
    Sustainability  ↗
  7. Dobrin (2023)
    AI and writing
  8. Elbow (1998)
    Writing with power: Techniques for mastering the writing process
  9. Erito (2023)
    Exploring ESP students’ perception toward the potential of artificial intelligence to pro…
    Journal of English Language Learning
  10. Eyman (2015)
    Digital rhetoric: Theory, method, practice
  11. FLOWER et al. (1984)
    Images, Plans, and Prose: The Representation of Meaning in Writing
    Written Communication
  12. Gagich (2020)
    An introduction to and strategies for multimodal composing
    Writing spaces: Readings on writing
  13. Hafner (2020)
    Assessing digital multimodal composing in second language writing: Towards a process-base…
    Journal of Second Language Writing  ↗
  14. Haraway (1991)
    Simians, cyborgs, and women
  15. Jiang (2024)
    Emotions are what will draw people in”: A study of critical affective literacy through di…
    Journal of Adolescent & Adult Literacy  ↗
  16. Jiang (2024)
    Composing to Enact Affective Agency: Engaging Multimodal Antiracist Pedagogy in…
    College Composition and Communication
  17. Jiang (2019)
    Multimodal design and social advocacy: Charting future directions for design as an interdisciplinary engagement
  18. Jiang (2024)
    Toward a “more-than-digital” AI literacy: Reimagining agency and authorship in the postdi…
    Postdigital Science and Education
  19. Kang (2023)
    Beyond ChatGPT: Multimodal generative AI for L2 writers
    Journal of Second Language Writing  ↗
  20. Knowles (2024)
    Machine-in-the-loop writing: Optimizing the rhetorical load
    Computers and Composition
  21. Kress (2009)
    Multimodality: A social semiotic approach to contemporary communication
  22. Kress (2021)
    Reading images: The grammar of visual design
  23. Krueger (2014)
    Focus groups: A practical guide for applied research
  24. Leander (2020)
    Critical literacy for a posthuman world: When people read, and become, with machines
    British Journal of Educational Technology  ↗
  25. Leverenz (2014)
    Design Thinking and the Wicked Problem of Teaching Writing
    Computers and Composition
  26. Li (2024)
    A “Dance of storytelling”: Dissonances between substance and style in collabora…
    Computers and Composition
  27. Liu (2024)
    Investigating students’ cognitive processes in generative AI-assisted digital multimodal …
    Computers & Education  ↗
  28. Malik (2023)
    Exploring artificial intelligence in academic essay: Higher education student's perspective
    International Journal of Educational Research Open  ↗
  29. Marback (2009)
    Embracing Wicked Problems: The Turn to Design in Composition Studies
    College Composition and Communication
  30. Murray (2009)
    Non-discursive rhetoric: Image and affect in multimodal composition
  31. Ng (2021)
    Conceptualizing AI literacy: An exploratory review
    Computers and Education: Artificial Intelligence
  32. Palmeri (2012)
    Remixing composition: A history of multimodal writing pedagogy
  33. Pigg (2024)
    Research writing with ChatGPT: A descriptive embodied practice framework
    Computers and Composition
  34. Purdy (2014)
    What Can Design Thinking Offer Writing Studies?
    College Composition and Communication
  35. Rittel (1973)
    Dilemmas in a general theory of planning
    Policy Sciences  ↗
  36. Saldaña (2021)
    The coding manual for qualitative researchers
  37. Tham et al. (2024)
    Understanding Writing Instructors’ Feelings toward the Affordances of Multimoda…
    College Composition and Communication
  38. Takayoshi (2018)
    The material, embodied practices of composing with technologies
    The Routledge handbook of digital writing and rhetoric
  39. Tham et al. (2022)
    Extending Design Thinking, Content Strategy, and Artificial Intelligence into T…
    Journal of Technical Writing and Communication
  40. van Leeuwen (2017)
    Multimodal literacy
    Metaphor
  41. (2023)
    TextGenEd: An introduction to teaching with text generation technologies
  42. Vetter et al. (2024)
    Towards a framework for local interrogation of AI ethics: A case study on text …
    Computers and Composition
  43. Wickman (2014)
    Wicked Problems in Technical Communication
    Journal of Technical Writing and Communication
  44. Yan (2023)
    Impact of ChatGPT on learners in a L2 writing practicum: An exploratory investigation
    Education and Information Technologies  ↗
  45. Zhang (2024)
    ChatGPT: Enhancing learner agency and multilingual practices in English writing class
    Journal of Education, Humanities and Social Sciences  ↗
CrossRef global citation count: 36 View in citation network → Build reading path →

Related Articles

  1. Pedagogy Oct 2025
    Contributors ↗
    modern rhetorical theory rhetorical criticism genre theory cultural rhetorics first-year composition writing pedagogy advanced composition creative writing writing across the curriculum graduate education two-year college service learning teacher development technical communication professional writing labor and working conditions archival research multimodality artificial intelligence literacy studies race and writing gender and writing disability studies literary studies editorial matter
  2. College Composition and Communication Sep 2025
    Using the AI Life Cycle to Unblackbox AI Tools: Teaching Résumé 2.0 with Résumé Analytics and Computational Job-Résumé Matching ↗
    Huiling Ding
    writing pedagogy teacher development professional writing labor and working conditions qualitative research artificial intelligence literacy studies editorial matter
  3. Computers and Composition Mar 2025
    Multimodal composing with generative AI: Examining preservice teachers’ processes and perspectives ↗
    Blaine E. Smith; Amanda Yoshiko Shimizu; Sarah K. Burriss; Melanie Hundley; Emily Pendergrass
    writing pedagogy teacher development multimodality artificial intelligence literacy studies
  4. Computers and Composition Dec 2024
    “Wayfinding” through the AI wilderness: Mapping rhetorics of ChatGPT prompt writing on X (formerly Twitter) to promote critical AI literacies ↗
    Anuj Gupta; Ann Shivers-McNair
    rhetorical criticism writing pedagogy teacher development qualitative research digital rhetoric social media artificial intelligence literacy studies community literacy
  5. Pedagogy Oct 2024
    Contributors ↗
    rhetorical criticism first-year composition writing pedagogy basic writing writing across the curriculum graduate education two-year college service learning teacher development revision argument collaborative writing assessment writing program administration multimodality multilingual writers literacy studies race and writing disability studies community literacy editorial matter