Virtual Reality and Embodiment in Multimodal Meaning Making

Kathy A. Mills Australian Catholic University ; Laura Scholes Australian Catholic University ; Alinta Brown Australian Catholic University

Abstract

Immersive virtual reality (VR) technology is becoming widespread in education, yet research of VR technologies for students’ multimodal communication is an emerging area of research in writing and literacies scholarship. Likewise, the significance of new ways of embodied meaning making in VR environments is undertheorized—a gap that requires attention given the potential for broadened use of the sensorium in multimodal language and literacy learning. This classroom research investigated multimodal composition using the virtual paint program Google Tilt Brush™ with 47 elementary school students (ages 10–11 years) using a head-mounted display and motion sensors. Multimodal analysis of video, screen capture, and think-aloud data attended to sensory-motor affordances and constraints for embodiment. Modal constraints were the immateriality of the virtual text, virtual disembodiment, and somatosensory mismatch between the virtual and physical worlds. Potentials for new forms of embodied multimodal representation in VR involved extensive bodily, haptic, and locomotive movement. The findings are significant given that research of embodied cognition points to sensorimotor action as the basis for language and communication.

Journal
Written Communication
Published
2022-07-01
DOI
10.1177/07410883221083517
CompPile
Search in CompPile ↗
Open Access
OA PDF Hybrid
Topics
Export

Citation Context

Cited by in this index (4)

  1. Computers and Composition
  2. Written Communication
  3. IEEE Transactions on Professional Communication
  4. Research in the Teaching of English

References (62) · 2 in this index

  1. 10.2307/j.ctt4cgnxt
  2. 10.17265/2159-5313/2016.09.003
  3. National statement on ethical conduct in human research
  4. Social Alternatives
  5. 10.1007/BF03651935
Show all 62 →
  1. 10.1037/0096-3445.136.4.539
  2. The kineikonic mode: Towards a multimodal approach to moving image media
  3. 10.1016/j.tate.2017.11.010
  4. 10.1353/lib.2018.0001
  5. 10.1007/BF03651936
    Australian Journal of Language and Literacy  
  6. 10.18870/hlrc.v7i1.373
  7. 10.1080/17439884.2021.1928695
  8. 10.1590/S1517-97022011000200003
  9. 10.1017/CBO9780511805844
  10. Writing, literacies, and education in digital cultures
  11. 10.1007/s10648-015-9334-2
  12. 10.1007/s11423-020-09912-z
  13. Encyclopedia of research design
  14. The Routledge handbook of multimodal analysis
  15. Written Communication
  16. The benefits of using virtual reality in education: A comparison study
  17. 10.1016/j.compedu.2016.12.009
  18. 10.1007/s10639-017-9676-0
  19. 10.4324/9780203964101
  20. Themes in Science and Technology Education
  21. 10.1016/S0079-7421(08)60027-4
  22. Reading images: The grammar of visual design
  23. 10.1386/eta.7.3.233_1
  24. 10.1007/s11423-021-09960-z
  25. Qualitative research with text, image, and sound
  26. Digitizing literacy: Reflections on the haptics of writing
  27. 10.1207/S15326985EP3801_6
  28. 10.1057/9781137539724_10
  29. 10.1080/17439884.2021.1952428
  30. English Teaching: Practice and Critique
  31. Mills K. A., Exley B. (2022). Sensory literacies: The full sensorium in literacy learning. In Tierney R., Riv…
  32. Writing, literacies, and education in digital cultures
  33. 10.4324/9781003137368
  34. 10.3102/00346543076003317
  35. Multiliteracies: Literacy learning and the design of social futures
  36. 10.4324/9780203379493
  37. Reading visual narratives: Image analysis of children’s picture books
  38. 10.3357/AMHP.4596.2016
  39. 10.3138/9781487579548
  40. 10.7861/fhj.2019-0036
  41. 10.1016/j.compedu.2019.103778
  42. 10.1109/TVCG.2015.2403312
  43. 10.1145/2835966.2836288
  44. 10.1007/s12564-016-9467-0
  45. 10.2466/pms.2003.97.3.697
  46. 10.1080/17439884.2016.1170034
  47. The new handbook of multisensory processing
  48. Embodied cognition
  49. 10.1186/s41235-018-0092-9
  50. Body, language and mind
  51. 10.1007/978-3-319-22888-4_24
  52. 10.3200/PSFL.52.1.56-58
  53. Written Communication
  54. 10.18178/ijlt.3.1.33-37
  55. 10.1007/BF03651921
    Australian Journal of Language and Literacy  
  56. 10.3758/BF03196322
  57. 10.1007/s11423-018-9604-z