Abstract

In many literacy studies, it is important to establish the reliability of independent observers' judgments. Reliability most commonly is measured either by the percentage of agreement or the correlation between the observers' judgments. This article argues that the percentage of agreement measure is more difficult to interpret than are correlation measures because of the following: (a) the effects of chance agreement are not accounted for automatically by the percentage of agreement measure; and (b) rates of chance agreement are strongly influenced by the variability of the data, by “ceiling” and “floor” effects, and by the scoring of near agreement as perfect agreement. For these reasons, the authors recommend that the field of literacy research adopt correlation as the standard method for estimating the reliability of observers' judgments.

Journal
Written Communication
Published
1999-07-01
DOI
10.1177/0741088399016003004
Open Access
Closed
Topics

Citation Context

Cited by in this index (1)

  1. Written Communication

Cites in this index (15)

  1. Written Communication
  2. Written Communication
  3. Written Communication
  4. Written Communication
  5. Research in the Teaching of English
Show all 15 →
  1. Research in the Teaching of English
  2. Written Communication
  3. Written Communication
  4. Written Communication
  5. Research in the Teaching of English
  6. Research in the Teaching of English
  7. Written Communication
  8. Written Communication
  9. Research in the Teaching of English
  10. Research in the Teaching of English
Also cites 17 works outside this index ↓
  1. 10.1016/1075-2935(95)90005-5
  2. 10.2307/747620
  3. 10.2307/748198
  4. 10.2307/748030
  5. 10.1598/RRQ.31.1.4
  6. 10.1016/1075-2935(95)90013-6
  7. 10.1598/RRQ.31.3.5
  8. 10.2307/747631
  9. 10.1016/S1075-2935(96)90004-5
  10. 10.1207/s15326977ea0101_4
  11. 10.1598/RRQ.31.4.4
  12. 10.1598/RRQ.31.3.4
  13. 10.37514/JBW-J.1997.16.1.06
    Journal of Basic Writing  
  14. 10.1598/RRQ.31.4.5
  15. 10.1207/s15326977ea0402_2
  16. 10.2307/747624
  17. 10.1207/s15326977ea0202_1
CrossRef global citation count: 43 View in citation network →