• Year 7 students can learn to code engaging animated narratives with basic Scratch. • English teachers can learn to sufficient coding to support students coding stories. • Student animated narratives of 2 – 3 min can meet English curricula requirements. • Student multimodality use can be evaluated using a criterion-based framework. • Student coding proficiency can be extended through coding animated narratives. Coding animated stories in the English classroom has been advocated from over a decade ago as an integrated curriculum context for early teaching of computer programming while simultaneously developing students’ multimodal narrative authoring. However, related research has not adequately addressed English curriculum requirements for narrative creation. This article describes the development of a framework for analysing coded animated stories from the perspective of English curriculum expectations. Analysis of 23 stories showed substantial variation in the emphasis given to different multimodal resources among those stories with the most extensive use of such resources. Stories with limited use of these resources excluded those expressing characters’ emotions and positioning the audience to experience the story from a variety of points of view. Stories with extensive multimodal expression were at the upper, but not necessarily highest, coding proficiency levels, while some with high coding proficiency showed limited use of multimodal resources. Implications are drawn for coding as an engaging creative tool in English classrooms.

In place of (or as a complement to) “user experience research,” we propose “reader experience research” as a technique for tracing how contemporary readers make meaning through a host of social-semiotic modes. Significantly, such modes are always already conditioned by cognitive, social, economic, and technological factors. To illustrate how reader experience research can account for such factors, we describe the emergence of our institution’s Reader Experience Lab. We illustrate through three experiences how the lab (and reader experience research, in general) offers opportunities for gaining insight into how contemporary readers make meaning in and/or despite what Dan Keller terms a “culture of acceleration.”

People working at the intersection of composition and user experience often serve as the connective material that binds content to use. In merging fundamental skills of both in multimodal UX, practitioners position themselves as essential mediators connecting technical information, storytelling, and technologies that carry impactful messages across disciplines, audiences, and contexts. Building on previous work that advocates for the power of narrative in AR/VR storytelling, we demonstrate how combining the composing strategy of narrative layering with user testing can guide the creation of inclusive, community-centered VR experiences. To illustrate the power of this capacity, we ground our analysis in the design of a Virtual Reality experience about advanced water purification, outlining a method for how narrative layering and UX testing can be woven together to address a variety of perspectives through interdisciplinary, layered storytelling. In doing so, we argue that multimodal UX is most powerful when it blends the needs of a range of audiences to build stories that communicate complex information in an inclusive and engaging way.

User experience (UX) as both a vocation and a skillset is currently in the center of a wicked knot: emerging technologies such as generative artificial intelligence (GenAI) and large language models (LLMs) are (for the moment) widely accessible in unprecedented ways and are already heavily integrated into modern workplace practices and educational spaces. Further, workplace demands have led to a change in perception of the function and value of UX, and the field is facing new obstacles to hiring and research funding. Our article argues that a resituation of UX is needed: we-as instructors and administrators-need to focus on UX as an act of slow, embodied, and multimodal UX composition. To do this work, we offer the strategy of détournement as central to UX curriculum and preparing students for design work in a variety of rhetorical situations, expressed through our example assignments for instructors to implement within the college classroom.

Content personalization or tailoring content as per the needs of users has been a focus of technical communicators’ work since a very long time. Recently, algorithms have helped trace users’ characteristics such as devices they use, platforms they work on, local language spoken, etc. to personalize content through strategies like responsive content, automatic translation and so on. AI tools have extended algorithmic capabilities for personalization, but at the same time increased the randomness of personalized content. That is, algorithms produce different results for the same user at different times or different results for different users at the same time with the same prompt thus shifting the agency of both rhetors (or content creators) and the audience (or content users). While conventional technical communication pedagogy has focused on writing for users, and more recently on writing for algorithms which serve the users, today it is crucial to understand how technologies like AI impact knowledge consumption processes from a user experience perspective? And how can we teach content personalization and adaptive techniques in the increasingly digital spaces of audience interactions? These questions motivated our research. To follow the roles of algorithms and technical communicators closely, we analyzed three different case studies where algorithms are responsible for a high level of personalization beyond the decisions made by technical communicators. Our findings suggest that we must teach students to investigate concepts such as user personas in UX for understanding audiences, several methods of decision-making for content assets, and rhetorical ecology for a holistic view of content production to dissemination.

• Composing process focus including screen recording and think aloud protocols. • 11 explicit composing process activities are defined and described. • Establishes explicit metalanguage for digital multimodal composing teaching & research. • Includes illustrations of complexity of multimodal composing. Daily writing practices occur in digital environments and are often multimodal. Studies have attempted to interpret composing processes in these environments through text-based lenses and findings have yet to explicitly or effectively define and illustrate the complexities. This case study explores processes and activities of 5th-grade students as they compose using digital tools, multimodal resources, and navigate the opportunities those tools and resources afford. Findings suggest 11 process activities; three unique to digital multimodal environments, and all having influences of the digital and multimodal environments in which composing takes place. Results 1) demonstrate the potential to develop a specific metalanguage for digital multimodal composing, 2) begin to inform a specific digital lens for interpreting composing in these 21 st century environments and 3) help practitioners design instruction that best support student composers in classroom contexts.

The question of how generative Artificial Intelligence (Gen AI) will reshape communication is causing questions and concerns across the field of education, particular literacy and writing classrooms. Although important questions have surfaced surrounding the varied effects on writing instruction and ethical implications of AI in the classroom, there are calls for deeper investigations about how these tools might shape multimodal composing processes. This study builds upon this developing field by exploring how 21 university students in literacy education courses multimodally composed with generative AI and their perspectives on the use of AI in the classroom. Data sources included screen capture and video observations, design interviews, pre- and post- surveys, and multimodal products. Through qualitative and multimodal analysis, four main themes emerged for understanding preservice teachers’ multimodal composing processes: (1) composing was an iterative process of prompting guided by the AI tools, (2) composers exhibited two distinct processes when designing their projects, (3) AI shaped creative possibilities, and (4) play, humor, and surprise served a key function while composing. Preservice teachers’ perspectives also revealed insights into how AI shaped engagement with content, the importance of scaffolding AI in the classroom, and how ethics were intertwined with technical function and teaching beliefs.

• The informality of video essay narration engendered ideation in drafting and script writing for students. • Students felt more responsible and personally invested in their arguments when they narrated and dramatized them in the video making process. • While students admitted that they tended to “gloss over” written drafts when revising, the video making process prompted students to be more self-motivated in the revising process, enabling them to evaluate and develop their arguments. • Unlike oral presentations, as students viewed their video essays as audience members, they could more clearly discern if their arguments lacked coherence or depth. This self-evaluation resulted in students taking the initiative to revise their final written assignments. Although multimodal assignments have increasingly been incorporated into academic writing curricula, research into their impact on student writing remains limited. This study, conducted at a Singaporean university, required students to transform a written essay draft into a video essay and then revise their draft into a written essay assignment. By comparing students’ initial drafts and their final submissions, and analysing interviews and reflective journals, we identified significant benefits stemming from the transmediation between written and multimodal text. Specifically, we found that 1) transmediation enabled students to self-evaluate their writing as they repeatedly listened to their voiceovers, found concrete visuals to illustrate their ideas, and edited their work to fit the concise video format; 2) students broke with habitual, less useful revision practices as they were freed from the conventional and grammatical concerns of written academic text and narrated their arguments colloquially in their voiceovers; 3) students exhibited an improved awareness of audience and medium; and 4) students were more enthusiastic with the course due to the novelty of the multimodal assignment. These findings suggest that including a video essay assignment during the drafting process can serve as an effective tool in advancing students’ abilities to evaluate their own academic writing.

• This study explores GenAI's role in multimodal composition, including Adobe Firefly and DALL·E. • GenAI reshapes the composition stages of invention, designing, and revising. • Despite its limitations, GenAI offers alternative solutions to wicked problems. • Post-GenAI use, students critically revise and iterate their compositions. • The study contributes to future research and teaching of AI-assisted composition. This study explores the integration of generative artificial intelligence (GenAI) design technologies, including Adobe Firefly and DALL·E, into the teaching and learning of multimodal composition. Through focus group discussions and case studies, this paper demonstrates the potential of GenAI in reshaping the various stages of the composition process, including invention, designing, and revising. The findings reveal that GenAI technologies have the potential to enhance students’ multimodal composition practices and offer alternative solutions to the wicked problems encountered during the design process. Specifically, GenAI facilitates invention by offering design inspirations and enriches designing by expanding, removing, and editing the student-produced design contents. The students in this study also shared their critical stance on the revision process by modifying and iterating their designs after their uses of GenAI. Through showcasing both the opportunities and challenges of GenAI technologies, this paper contributes to the ongoing scholarly conversations on multimodal composition and pedagogy. Moreover, the paper offers implications for the future research and teaching of GenAI-assisted multimodal composition projects, with the aim of encouraging thoughtful integration of GenAI technologies to foster critical AI literacy among college composition students.

In this article, an analysis of multimodal representations in elementary school students’ descriptive texts about the mobile game Pokémon Go (PG) is used to discuss youths’ new literacy practices emerging from their out-of-school experiences (e.g., gaming, producing game-walkthroughs, and fan art). Social semiotic multimodal analyses of two students’ PG texts and their participation in talks around their texts are used to exemplify what occurs semiotically in the translation of meanings and designs across modes, media, and sites. Combining the analysis of the meaning potentials of multimodal representations with ethnographic accounts of their use in context produces the following findings: The PG writing task connects with the students’ life-worlds and the gaming context around PG prompts the design of their multimodal representations. The students are active creators of content and demonstrate purposeful and innovative uses of semiotic resources in self-representation, stance taking, and audience awareness. The iPad facilitates multimodal and digital crossovers between leisure activities and school subjects. The concluding discussion suggests how game-based literacy practices could be transferred into academic settings.