IEEE Transactions on Professional Communication Jan 2005

A framework for analyzing levels of analysis issues in studies of e-collaboration

M.J. Gallivan ; R. Benunan-Fich

Abstract

There has been a proliferation of competing explanations regarding the inconsistent results reported by the e-collaboration literature since its inception. This study advances another possible explanation by investigating the range of multilevel issues that can be encountered in research on the use of synchronous or asynchronous group support systems. We introduce concepts of levels of analysis from the management literature and then examine all empirical studies of e-collaboration from seven information systems journals for the period 1999-2003. We identified a total of 54 studies of e-collaboration in these journals, and after excluding 18 nonconforming studies - those that were primarily conceptual, qualitative, or exploratory only-we analyzed the levels of analysis issues in the remaining 36 empirical studies. Based on our analysis and classification of these studies into six different clusters according to their levels of analysis, we found that a majority of these studies contain one or more problems of levels incongruence that cast doubts on the validity of their results. It is indeed possible that these methodological problems are in part responsible for the inconsistent results reported in this literature, especially since researchers' frequent decisions to analyze data at the individual level - even when the theory was formulated at the group level and when the research setting featured individuals working in groups -may very well have artificially inflated the authors' chances of finding statistically significant results. Based on our discussion of levels of analysis concepts, we hope to provide guidance to empirical researchers who study e-ollaboration.

Journal
IEEE Transactions on Professional Communication
Published
2005-01-01
DOI
10.1109/tpc.2005.843301
CompPile
Open Access
Closed
Topics
collaborative writing empirical research online writing instruction
Export
BibTeX RIS

Citation Context

Cited by in this index (1)

  1. Koh et al. (2012)
    Too Early, Too Bad: Uncovering and Understanding the Initial Participation Para…
    IEEE Transactions on Professional Communication

References (106) · 1 in this index

  1. 10.7312/luca94612
     ↗
  2. 10.1080/07421222.2000.11518272
     ↗
  3. Roach (1987)
    America’s technology dilemma: A profile of the information economy
  4. Roach (1991)
    Services under siege: The restructuring imperative
    Harvard Bus. Rev.
  5. 10.1145/163298.163309
     ↗
Show all 106 →
  1. 10.1093/oso/9780195068061.003.0004
     ↗
  2. 10.1287/mnsc.42.4.541
     ↗
  3. 10.1257/jep.14.4.23
     ↗
  4. 10.1145/641865.641866
     ↗
  5. 10.2307/249475
     ↗
  6. 10.1287/isre.10.2.167
     ↗
  7. Soh
    How IT creates business value: A process theory synthesis
  8. 10.1287/isre.6.1.3
     ↗
  9. 10.1162/003355302753399526
     ↗
  10. 10.1287/isre.10.2.110
     ↗
  11. 10.2307/3250972
     ↗
  12. 10.1287/mnsc.34.5.583
     ↗
  13. 10.1080/07421222.2002.11045728
     ↗
  14. 10.2307/3250928
     ↗
  15. Rousseau (1985)
    Issues in level in organizational research: Multi level and cross level perspectives
  16. 10.2307/2392578
     ↗
  17. 10.2307/258703
     ↗
  18. Klein (2000)
    Multilevel Theory, Research, and Methods in Organizations
  19. 10.5465/AMR.1999.1893934
     ↗
  20. Kohli
    In search of IT business value: Do measurement levels make a difference?
  21. 10.2307/258559
     ↗
  22. 10.1080/07421222.2002.11045696
     ↗
  23. 10.1080/07421222.1998.11518216
     ↗
  24. 10.1080/07421222.2000.11045657
     ↗
  25. 10.1287/orsc.5.2.121
     ↗
  26. 10.2307/249668
     ↗
  27. 10.1287/isre.8.4.342
     ↗
  28. 10.1287/isre.13.1.91.93
     ↗
  29. 10.1023/A:1008765029795
     ↗
  30. Klein (2000)
    Multilevel analytical techniques: Commonalities, differences, and continuing questions
  31. 10.1037/0021-9010.86.1.3
     ↗
  32. 10.1037/0021-9010.77.2.161
     ↗
  33. Kozlowski (2000)
    A multilevel approach to theory and research in organizations
  34. 10.2307/259086
     ↗
  35. Roberts (1978)
    Developing an Interdisciplinary Science of Organizations
  36. 10.1037/0021-9010.67.2.219
     ↗
  37. 10.1037//0021-9010.69.1.85
     ↗
  38. Bliese (2000)
    Within-group agreement, nonindependence, and reliability: Implications for data aggregati…
  39. 10.1177/014920639802400202
     ↗
  40. 10.1016/S1048-9843(01)00105-9
     ↗
  41. House
    The meso paradigm: A framework for integration of micro and macro organizational behavior
  42. 10.1037/0021-9010.78.2.306
     ↗
  43. 10.1177/014920630202800305
     ↗
  44. 10.2307/259082
     ↗
  45. Kock (2003)
    Action research: Lessons learned from a multi-iteration study of computer-media…
    IEEE Transactions on Professional Communication
  46. 10.2307/4132339
     ↗
  47. 10.1287/isre.10.2.186
     ↗
  48. 10.1080/07421222.2000.11045630
     ↗
  49. 10.1287/isre.12.3.304.9708
     ↗
  50. 10.1287/isre.11.1.61.11783
     ↗
  51. 10.2307/30036518
     ↗
  52. Yoris (2001)
    Should we wait? Network externalities, compatibility, and electronic billing adoption
    J. Manage. Inform. Syst.  ↗
  53. 10.1080/07421222.2002.11045721
     ↗
  54. 10.1057/palgrave.ejis.3000375
     ↗
  55. 10.1057/palgrave.ejis.3000455
     ↗
  56. 10.1016/S0167-9236(99)00049-4
     ↗
  57. 10.1016/S0167-9236(99)00042-1
     ↗
  58. 10.1016/S0167-9236(02)00132-X
     ↗
  59. Briggs (1998/1999)
    A technology transition model derived from field investigation of GSS use
    J. Manage. Inform. Syst.  ↗
  60. Briggs (2003)
    Collaboration engineering with thinklets to pursue sustained success with group support systems
    J. Manage. Inform. Syst.  ↗
  61. 10.2307/3250948
     ↗
  62. 10.2307/3250930
     ↗
  63. Scott (2000)
    Facilitating interorganizational learning with information technology
    J. Manage. Inform. Syst.  ↗
  64. 10.1080/07421222.1998.11518218
     ↗
  65. 10.1080/07421222.2003.11045742
     ↗
  66. 10.2307/3250979
     ↗
  67. 10.1080/07421222.2003.11045768
     ↗
  68. 10.1016/S0167-9236(02)00124-0
     ↗
  69. 10.1016/S0167-9236(02)00072-6
     ↗
  70. 10.1080/07421222.2002.11045691
     ↗
  71. 10.2307/30036538
     ↗
  72. 10.2307/3250922
     ↗
  73. 10.1080/07421222.2000.11045647
     ↗
  74. 10.1016/S0167-9236(02)00009-X
     ↗
  75. 10.1080/07421222.2003.11045754
     ↗
  76. 10.1080/07421222.1999.11518238
     ↗
  77. 10.1080/07421222.2002.11045697
     ↗
  78. 10.1287/isre.11.4.386.11874
     ↗
  79. 10.1057/palgrave.ejis.3000335
     ↗
  80. 10.1046/j.1365-2575.2001.00103.x
     ↗
  81. 10.2307/249489
     ↗
  82. 10.1080/07421222.1999.11518260
     ↗
  83. 10.1016/S0167-9236(01)00143-9
     ↗
  84. 10.1287/isre.12.3.322.9710
     ↗
  85. Hilmer (2000)
    Stimulating thinking: Cultivating better decisions with groupware through categorization
    J. Manage. Inform. Syst.  ↗
  86. 10.1016/S0167-9236(99)00028-7
     ↗
  87. 10.1080/10864415.2003.11044295
     ↗
  88. 10.2307/3250989
     ↗
  89. 10.1287/isre.13.4.404.72
     ↗
  90. 10.2307/249411
     ↗
  91. Dennis (2003)
    The adoption and use of GSS in project teams: Toward more participative processes and outcomes
    MIS Quart.  ↗
  92. Hayne (2003)
    Identification of comment authorship in anonymous group support systems
    J. Manage. Inform. Syst.  ↗
  93. Hender (2002)
    An examination of the impact of stimuli type and GSS structure on creativity: Brainstormi…
    J. Manage. Inform. Syst.  ↗
  94. Karahanna (2002)
    Individual differences and relative advantage: The case of GSS
    Decision Support Syst.  ↗
  95. 10.1057/palgrave.ejis.3000421
     ↗
  96. 10.1080/07421222.2002.11045731
     ↗
  97. 10.1057/palgrave.ejis.3000358
     ↗
  98. 10.1080/07421222.1999.11518223
     ↗
  99. 10.1080/07421222.2003.11045750
     ↗
  100. 10.1287/isre.13.1.70.92
     ↗
  101. 10.1046/j.1365-2575.1999.00065.x
     ↗
CrossRef global citation count: 7 View in citation network → Build reading path →

Related Articles

  1. IEEE Transactions on Professional Communication Mar 2005
    A Framework for Analyzing Levels of AnalysisIssues in Studies of E-Collaboration ↗
    M.J. Gallivan; R. Benbunan-Fich
    collaborative writing empirical research online writing instruction
  2. Written Communication May 2026
    Beyond Co-Regulation: Interplay as a Methodological Framework for Examining Self-Regulation in Generative AI-Assisted Writing ↗
    Ryan Roderick; Susan Tanner
    writing pedagogy collaborative writing assessment empirical research artificial intelligence
  3. IEEE Transactions on Professional Communication Mar 2026
    Bridging the Gap: A Comparative Study of Students’ and LSPs’ Perceptions of Translation Internships ↗
    Shuang Liu
    collaborative writing assessment professional writing empirical research
  4. Computers and Composition Dec 2025
    Supporting online learning for diverse elementary students: A community of inquiry approach to collaborative multimodal composing—processes, products, and perspectives ↗
    Amanda Yoshiko Shimizu; Jill Santos
    collaborative writing multimodality online writing instruction
  5. Computers and Composition Sep 2025
    When collaborating turns into dishonesty: A data-driven heuristic comparing human and AI collaborators ↗
    John R. Gallagher; Kyle Wagner; Jordan Canzonetta
    collaborative writing empirical research artificial intelligence