Holistic reading is widely used to assess the proficiency of non-native-speaking (NNS) writers. However, ESL professionals, who have been profoundly influenced by the notion that attention to the NNS author's message is an integral part of teaching the writing process, have questioned how well native-speaking (NS) raters comprehend NNS texts, given that the task of decoding NNS prose is even further complicated by the time constraints of the holistic scoring process itself. This article describes a study that investigated the extent to which NS holistic raters comprehend NNS texts. After rating several practice compositions, subjects rated one of two qualitatively distinct essays, and then wrote recall protocols to test their comprehension. Data analysis revealed that readers of the better written text recalled significantly more than did readers of the less well written text, indicating that NS holistic raters attend to meaning when evaluating NNS writing proficiency.