Research problem: Concurrent think-aloud (CTA) protocols are one of the dominant approaches of usability testing. However, there is still debate about the validity of the method, partly focusing on the usefulness and exhaustiveness of participants' verbalizations. The rise of eye-tracking technology sheds new light on this discussion, as participants' working processes can now be observed in more detail. Research questions: (1) What kinds of verbalizations do participants produce, and how do they relate to the information that can be directly observed using eye tracking? (2) What do eye movements reveal about cognitive processes at times when participants stop verbalizing? Literature review: Our study replicates an earlier study by Cooke (2010), who used a combination of CTA protocols and eye tracking in a small sample with experienced and highly educated participants to investigate the validity of CTA. Cooke's results suggest that the additional value of participants' verbalizations is limited: at least 77% of the verbalizations referred to things that could be easily observed with eye tracking. Methodology: We conducted a study in which 60 participants with different characteristics performed tasks on informational websites. During their task performance, they verbalized their thoughts, and simultaneously their eye movements were measured. The resulting think-aloud protocols were divided in verbalization units, which were coded into content types. Silences were registered, and eye movements during these silences were analyzed. Results and discussion: We found a different distribution of verbalization types than Cooke (2010) reported, with far more verbalizations where participants formulated doubts, judgments on the website, or expressions of frustration. In our study, verbalizations provided a substantial contribution in addition to the directly observable user problems. We measured a rather high percentage of silences (27%), during which participants most often were scanning pages for information. During these silences, interesting observations could be made about users' processes and obstacles on the website. The implication of our study is that we now have a better understanding of the types of verbalizations that a CTA evaluation might generate. Further, we know that relevant usability observations can be made during silences. A limitation is that we do not know yet the influence of specific characteristics of the evaluation setting on the types of verbalizations and silences. Future research should focus on the influence of evaluation settings on the outcomes of an evaluation, in particular, the influence of characteristics of the participants who are involved in the study.

Web sites increasingly encourage users to provide comments on the quality of the content by clicking on a feedback button and filling out a feedback form. Little is known about users’ abilities to provide such feedback. To guide the development of evaluation tools, this study examines to what extent users with various background characteristics are able to provide useful comments on informational Web sites. Results show that it is important to keep the feedback tools both simple and attractive so that users will be able and willing to provide useful feedback on Web site pages.

Reader feedback is generally considered to be valuable input for writers who want to optimize their documents, but a reader-focused evaluation is often time-consuming. For this reason, we have developed Focus, a software tool for collecting reader comments more efficiently. The design and rationale of the software are described in this article. In a small-scale evaluation study, the results we obtained using Focus were compared to the reader feedback collected under the plus-minus method. It appeared that the number of problems detected per participant did not differ, but there were differences in the types of problems found. Focus participants appeared to comment more from a reviewer's and less from a user's perspective. Although the two methods are not interchangeable, Focus can be said to be a promising evaluation tool, deserving further research.

How have Dutch instructive texts changed in the course of the last century? This question is the topic of a research project presented in this article. First, we give some insight into the kind of documents we have collected in our corpus. The oldest instructive texts date from the beginning of the nineteenth century. But for most technical devices, the tradition of adding an instructive text starts about 1925. After that we present a few results of the investigations: the disappearance of persuasive passages and of realistic representations of human beings. Finally, we give a more detailed sketch of the development of the algorithmic style.