Locating the Semiotic Power of Multimodality

Glynda A. Hull ; Mark Evan Nelson University of California, Berkeley

Abstract

This article reports research that attempts to characterize what is powerful about digital multimodal texts. Building from recent theoretical work on understanding the workings and implications of multimodal communication, the authors call for a continuing empirical investigation into the roles that digital multimodal texts play in real-world contexts, and they offer one example of how such investigations might be approached. Drawing on data from the practice of multimedia digital storytelling, specifically a piece titled “Lyfe-N-Rhyme,” created by Oakland, California, artist Randy Young (accessible at http://www.oaklanddusty.org/videos.php), the authors detail the method and results of a fine-grained multimodal analysis, revealing semiotic relationships between and among different, copresent modes. It is in these relationships, the authors argue, that the expressive power of multimodality resides.

Journal
Written Communication
Published
2005-04-01
DOI
10.1177/0741088304274170
CompPile
Search in CompPile ↗
Open Access
Closed
Topics
Export

Citation Context

Cited by in this index (35)

  1. Writing and Pedagogy
  2. Research in the Teaching of English
  3. Written Communication
  4. Written Communication
  5. Written Communication
Show all 35 →
  1. Written Communication
  2. Research in the Teaching of English
  3. Research in the Teaching of English
  4. Computers and Composition
  5. Written Communication
  6. Written Communication
  7. Research in the Teaching of English
  8. Written Communication
  9. Rhetoric Review
  10. Written Communication
  11. Computers and Composition
  12. Written Communication
  13. Communication Design Quarterly
  14. Written Communication
  15. Technical Communication Quarterly
  16. Journal of Business and Technical Communication
  17. Computers and Composition
  18. Communication Design Quarterly
  19. Written Communication
  20. Written Communication
  21. Computers and Composition
  22. Computers and Composition
  23. Computers and Composition
  24. Written Communication
  25. Written Communication
  26. Written Communication
  27. Computers and Composition
  28. Technical Communication Quarterly
  29. Written Communication
  30. Written Communication

References (55) · 2 in this index

  1. Rewriting literacy: Culture and the discourses of the other
  2. Writing space: The computer, hypertext, and the history of writing
  3. The psychology of literacy
  4. Doing documentary work
  5. The social construction of literacy
Show all 55 →
  1. Multiliteracies
  2. Other people’s children
  3. Course in general linguistics
  4. 10.3102/0013189X032001005
  5. 10.1598/RRQ.39.1.3
  6. Discourse and social power
  7. Critical discourse analysis
  8. Communicating: The multiple modes of human interconnection
  9. Cultural studies
  10. What video games have to teach us about learning and literacy
  11. The ecological approach to visual perception
  12. Forms of talk
  13. Computers and Composition
  14. Global literacies and the World Wide Web
  15. Sound identities: Popular music and the cultural politics of education
  16. Blurring boundaries: Research and Teaching Beyond a Discipline
  17. Crafting an agentive self: Case studies on digital storytelling
  18. National Council of Teachers of English Research Assembly
  19. Before writing: Rethinking paths into literacy
  20. 10.4324/9780203201220_chapter_3
  21. 10.4324/9780203164754
  22. Reading images: A grammar of visual design
  23. Multimodal discourse. The modes and media of contemporary communication
  24. 10.1075/jnlh.7.49som
  25. Digital storytelling: Capturing lives, creating community
  26. Reading science
  27. Handbook of literacy and technology: Transformations in a post-typographic world
  28. Understanding media: The extensions of man
  29. An introduction to visual culture
  30. Conference on Visual Culture, University of California
  31. National assessment of adult literacy
  32. The neglected“R”: The need for a writing revolution
  33. Reading at risk: A survey of literary reading in America
  34. 10.17763/haer.66.1.17370n67v22j160u
  35. Living narratives: Creating lives in everyday storytelling
  36. Selected philosophical writings
  37. Selected philosophical writings
  38. 10.1525/9780520940888
  39. 10.3102/0013189X032001025
  40. 10.1177/0002764204266234
  41. 10.1017/CBO9781139524834.006
  42. Literacy in theory and practice
  43. The visual display of quantitative information
  44. Envisioning information
  45. Visual explanations: Images and quantities, evidence and narrative
  46. 10.1007/978-1-349-27700-1
  47. Mind in society: The development of higher psychological processes
  48. Thought and language
  49. Written Communication
  50. invited lecture at the University of Wisconsin-Madison