Although the usefulness of evaluating documents has become generally accepted among communication professionals, the supporting research that puts evaluation practices empirically to the test is only beginning to emerge. This article presents an overview of the available research on troubleshooting evaluation methods. Four lines of research are distinguished concerning the validity of evaluation methods, sample composition, sample size, and the implementation of evaluation results during revision.

The authors compare a reader-focused text evaluation with an expert-focused evaluation by technical writers and subject/audience experts. The experts were asked to predict the problems readers had signaled in a government brochure about alcohol. On average, they predicted less than 15% of the reader problems and produced a lot of new problem detections. In addition, the experts showed little mutual agreement in their problem detections. Their results suggest that a reader-focused evaluation should not be substituted for an expert-focused evaluation. The paper ends with a discussion of methodological issues for this type of research.