In the past years, comparative assessment approaches have gained ground as a viable method to assess text quality. Instead of providing absolute scores to a text as in holistic or analytic scoring methods, raters in comparative assessments rate text quality by comparing texts either to pre-selected benchmarks representing different levels of writing quality (i.e., benchmark rating method) or by a series of pairwise comparisons to other texts in the sample (i.e., comparative judgement; CJ). In the present study, text quality scores from the benchmarking method and CJ are compared in terms of their reliability, convergent validity and scoring distribution. Results show that benchmark ratings and CJ-ratings were highly consistent and converged to the same construct of text quality. However, the distribution of benchmark ratings showed a central tendency. It is discussed how both methods can be integrated and used such that writing can be assessed reliably, validly, but also efficiently in both writing research and practice.

To enable a proper evaluation of the results of writing interventions for scientific replication and theory building, it is of vital importance that the design principles underlying an intervention and operationalization thereof are clearly described. A detailed description of a writing intervention is also important from a practical point of view, to foster dissemination and successful implementation of the intervention into practice. In this paper we propose a framework for reporting on the design principles of multifaceted intervention programs in a systematic manner. Unique features of this framework are that we (1) separate the design principles for the focus and mode of instruction, (2) systematically describe how these principles are integrated and operationalized into learning and teaching activities, (3) systematically describe the professional development teachers need to be able to execute the teaching activities. We demonstrate how this framework can be applied, with a worked example of an intervention that we designed, implemented and tested in elementary schools in the Netherlands. The framework provided in this paper makes core features of writing interventions transparent to reviewers, other scholars, and educational practitioners, and warrants that an intervention includes all necessary elements in the most optimal way. Moreover, this type of framework facilitates the comparison of interventions across contexts and countries.