Written Communication Apr 2005

Locating the Semiotic Power of Multimodality

Glynda A. Hull ; Mark Evan Nelson University of California, Berkeley

Abstract

This article reports research that attempts to characterize what is powerful about digital multimodal texts. Building from recent theoretical work on understanding the workings and implications of multimodal communication, the authors call for a continuing empirical investigation into the roles that digital multimodal texts play in real-world contexts, and they offer one example of how such investigations might be approached. Drawing on data from the practice of multimedia digital storytelling, specifically a piece titled “Lyfe-N-Rhyme,” created by Oakland, California, artist Randy Young (accessible at http://www.oaklanddusty.org/videos.php), the authors detail the method and results of a fine-grained multimodal analysis, revealing semiotic relationships between and among different, copresent modes. It is in these relationships, the authors argue, that the expressive power of multimodality resides.

Journal
Written Communication
Published
2005-04-01
DOI
10.1177/0741088304274170
Open Access
Closed
Topics
multimodality multilingual writers

Citation Context

Cited by in this index (32)

  1. Afolalu (2024)
    More than My Race: Deconstructing Racial Identity Categories through Digital Li…
    Research in the Teaching of English
  2. Lee (2023)
    Writing Toward a Decolonial Option: A Bilingual Student’s Multimodal Composing …
    Written Communication
  3. Reid et al. (2022)
    One Fourth-Grader’s Orchestration of Modes Through Comic Composition
    Written Communication
  4. Mills et al. (2022)
    Virtual Reality and Embodiment in Multimodal Meaning Making
    Written Communication
  5. Höglund (2022)
    The Heartbeat of Poetry: Student Videomaking in Response to Poetry
    Written Communication
Show all 32 →
  1. Marciano et al. (2022)
    Generating New Narratives: Examining Youths’ Multiliteracies Practices in Youth…
    Research in the Teaching of English
  2. Lam et al. (2021)
    Multimodal Voicing and Scale-Making in a Youth-Produced Video Documentary on Im…
    Research in the Teaching of English
  3. Castillo (2020)
    Mixing Tracks: Notes toward the analysis and design of vocal manipulation in Hi…
    Computers and Composition
  4. Ríos (2020)
    Writing Oneself Into the Curriculum: Photovoice Journaling in a Secondary Ethni…
    Written Communication
  5. Smirnov et al. (2019)
    “Presenting Our Perspective”: Recontextualizing Youths’ Experiences of Hypercri…
    Written Communication
  6. Smith (2019)
    Mediational Modalities: Adolescents Collaboratively Interpreting Literature thr…
    Research in the Teaching of English
  7. Reid (2019)
    Compressing, Expanding, and Attending to Scientific Meaning: Writing the Semiot…
    Written Communication
  8. Butler (2018)
    Integral Captions and Subtitles: Designing a Space for Embodied Rhetorics and V…
    Rhetoric Review
  9. Smith (2018)
    Composing for Affect, Audience, and Identity: Toward a Multidimensional Underst…
    Written Communication
  10. Jocius (2018)
    Becoming Entangled: An Analysis of 5th Grade Students Collaborative Multimodal …
    Computers and Composition
  11. Cimasko et al. (2017)
    Multimodal Resemiotization and Authorial Agency in an L2 Writing Classroom
    Written Communication
  12. Anderson et al. (2017)
    Seeing Academically Marginalized Students’ Multimodal Designs From a Position o…
    Written Communication
  13. Bourelle et al. (2015)
    Multimodality in the Technical Communication Classroom: Viewing Classical Rheto…
    Technical Communication Quarterly
  14. Wickman (2015)
    Locating the Semiotic Power of Writing in Science
    Journal of Business and Technical Communication
  15. Williams (2014)
    From Screen to Screen: Students’ Use of Popular Culture Genres in Multimodal Wr…
    Computers and Composition
  16. Oddo (2013)
    Discourse-Based Methods Across Texts and Semiotic Modes
    Written Communication
  17. Shanahan (2013)
    Composing “Kid-Friendly” Multimodal Text
    Written Communication
  18. Jwa (2012)
    Modeling L2 Writer Voice: Discoursal Positioning in Fanfiction Writing
    Computers and Composition
  19. Turnley (2011)
    Towards a Mediological Method: A Framework for Critically Engaging Dimensions o…
    Computers and Composition
  20. Vojak et al. (2011)
    New Spaces and Old Places: An Analysis of Writing Assessment Software
    Computers and Composition
  21. Gilje (2010)
    Multimodal Redesign in Filmmaking Practices: An Inquiry of Young Filmmakers’ De…
    Written Communication
  22. Vasudevan et al. (2010)
    Rethinking Composing in a Digital Age: Authoring Literate Identities Through Mu…
    Written Communication
  23. Wickman (2010)
    Writing Material in Chemical Physics Research: The Laboratory Notebook as Locus…
    Written Communication
  24. Purdy (2010)
    The Changing Space of Research: Web 2.0 and the Integration of Research and Wri…
    Computers and Composition
  25. Swarts et al. (2009)
    Guest Editors' Introduction: New Technological Spaces
    Technical Communication Quarterly
  26. Nelson et al. (2008)
    Challenges of Multimedia Self-Presentation
    Written Communication
  27. Ranker (2008)
    Composing Across Multiple Media
    Written Communication

Cites in this index (2)

  1. Haas (1999)
    On the relationship between old and new technologies
    Computers and Composition
  2. WITTE (1992)
    Context, Text, Intertext
    Written Communication
Also cites 12 works outside this index ↓
  1. Coles, R. (1997)
    Doing documentary work
     ↗
  2. 10.3102/0013189X032001005
     ↗
  3. 10.1598/RRQ.39.1.3
     ↗
  4. 10.4324/9780203201220_chapter_3
     ↗
  5. 10.4324/9780203164754
     ↗
  6. 10.1075/jnlh.7.49som
     ↗
  7. 10.17763/haer.66.1.17370n67v22j160u
     ↗
  8. 10.1525/9780520940888
     ↗
  9. 10.3102/0013189X032001025
     ↗
  10. 10.1177/0002764204266234
     ↗
  11. 10.1017/CBO9781139524834.006
     ↗
  12. 10.1007/978-1-349-27700-1
     ↗
CrossRef global citation count: 417 View in citation network →

Related Articles

  1. Journal of Technical Writing and Communication Apr 2026
    What Video Games Can Tell Us About Interactive Technical Communication ↗
    Emily Kuzneski Johnson
    multimodality technical communication multilingual writers
  2. Journal of Technical Writing and Communication Apr 2026
    “Is This Ethical?” New Data on the Ethical Principles and Practices of Document Design ↗
    Leslie Seawright; Amy Hodges; Timothy Ponce
    multimodality technical communication disability studies multilingual writers
  3. Journal of Technical Writing and Communication Mar 2026
    “It's Hard to Show ROI When You’re Preventing Things from Happening”: How Impact Storytelling Frames Community Health Initiatives for Executive Audiences ↗
    Margaret Hsiao
    qualitative research multimodality technical communication professional writing multilingual writers
  4. Written Communication Feb 2026
    Reading Medium and Communicative Purpose in Writing: Effects on Pausing Behaviour and Text Quality, Controlling for Reading Comprehension and Executive Functions ↗
    Ángel Valenzuela; Cristian A. Rojas-Barahona; Ramón D. Castillo; Ladislao Salmerón
    assessment curriculum design multimodality multilingual writers revision
  5. Literacy in Composition Studies Feb 2026
    All Are Connected: From Traditional Chinese Medicine to Students’ Literacy Practices Reviewing Doing Difference Differently: Chinese International Students’ Literacy Practices and Affordances by Zhaozhe Wang ↗
    Shi, Carina Jiaxing
    multimodality multilingual writers