IEEE Transactions on Professional Communication Jan 1997

The evaluation of text quality: expert-focused and reader-focused methods compared

L. Lentz ; M. De Jong

Abstract

The authors compare a reader-focused text evaluation with an expert-focused evaluation by technical writers and subject/audience experts. The experts were asked to predict the problems readers had signaled in a government brochure about alcohol. On average, they predicted less than 15% of the reader problems and produced a lot of new problem detections. In addition, the experts showed little mutual agreement in their problem detections. Their results suggest that a reader-focused evaluation should not be substituted for an expert-focused evaluation. The paper ends with a discussion of methodological issues for this type of research.

Journal
IEEE Transactions on Professional Communication
Published
1997-01-01
DOI
10.1109/47.649557
CompPile
Open Access
Closed
Topics
assessment
Export
BibTeX RIS

Citation Context

Cited by in this index (12)

  1. Fuller et al. (2021)
    Conceptualizing Empathy Competence: A Professional Communication Perspective
    Journal of Business and Technical Communication
  2. Schriver (2017)
    Plain Language in the US Gains Momentum: 1940–2015
    IEEE Transactions on Professional Communication
  3. Wiatrak-Uhlenkott et al. (2013)
    Book Reviews: Rhetorical Strategies and Genre Conventions in Literary Studies: …
    Journal of Technical Writing and Communication
  4. Jong et al. (2006)
    Dynamics of Iterative Reader Feedback: An Analysis of Two Successive Plus-Minus…
    Journal of Business and Technical Communication
  5. Grove (2003)
    In pursuit of constructive criticism
    IEEE Transactions on Professional Communication
Show all 12 →
  1. Jong et al. (2001)
    Focus: Design and Evaluation of a Software Tool for Collecting Reader Feedback
    Technical Communication Quarterly
  2. Jong et al. (2001)
    Readers' Background Characteristics and Their Feedback on Documents: The Influe…
    Journal of Technical Writing and Communication
  3. Jong et al. (2000)
    Toward a document evaluation methodology: what does research tell us about the …
    IEEE Transactions on Professional Communication
  4. Jong et al. (2000)
    Toward a document evaluation methodology: what does research tell us about the …
    IEEE Transactions on Professional Communication
  5. Lentz et al. (2000)
    Babel in document design: the evaluation of multilingual texts
    IEEE Transactions on Professional Communication
  6. Lentz et al. (2000)
    Babel in document design: the evaluation of multilingual texts
    IEEE Transactions on Professional Communication
  7. Lentz et al. (1997)
    The evaluation of text quality: expert-focused and reader-focused methods compared
    IEEE Transactions on Professional Communication

References (19) · 4 in this index

  1. de jong (1994)
    de pretest als schakel in het tekstontwerpproces
    Tijdschrift voor Taalbeheersing
  2. virzi (1992)
    refining the test phase of usability evaluation: how many subjects is enough?
    Human Factors  ↗
  3. baker (1985)
    how do we know when we don't understand? standards for evaluating text comprehension
    Metacognition Cognition and Human Performance
  4. 10.1007/BF01326548
     ↗
  5. 10.1201/9781420055948.ch7
     ↗
Show all 19 →
  1. schriver (1987)
    Teaching writers to anticipate the reader s needs Empirically based instruction
  2. mack (1994)
    observing, predicting and analyzing usability problems
    Usability Inspection Methods
  3. schriver (1992)
    teaching writers to anticipate readers' needs: what can document designers learn from usa…
    Studies of Functional Text Quality  ↗
  4. lentz (1993)
    Wat mankeert er aan die tekst? De evaluatie van voorlichtingsteksten over subsidieregelingen (What is wrong with that text? Evaluation of public information texts on subsidy regulations)
  5. schriver (1989)
    document design from 1980 to 1989: challenges that remain
    Tech Commun
  6. Lentz et al. (1997)
    The evaluation of text quality: expert-focused and reader-focused methods compared
    IEEE Transactions on Professional Communication
  7. swaney (1991)
    editing for comprehension: improving the process through reading protocols
    Plain Language Principles and Practice
  8. Jong et al. (1996)
    Expert Judgments versus Reader Feedback: A Comparison of Text Evaluation Techniques
    Journal of Technical Writing and Communication
  9. schriver (1995)
    Document design as rhetorical action
  10. de jong (1994)
    toward a valid design for pretesting and revising leaflets
    Functional Communication Quality  ↗
  11. de haas (1994)
    research for government information campaigns: not a final judgment but a management tool
    Government Information in The Netherlands
  12. dieli (1986)
    Designing successful documents An investigation of document evaluation methods
  13. Schriver (1989)
    Evaluating text quality: the continuum from text-focused to reader-focused methods
    IEEE Transactions on Professional Communication
  14. Vet (1996)
    Technical Writing in the Netherlands
    Journal of Technical Writing and Communication
CrossRef global citation count: 36 View in citation network → Build reading path →

Related Articles

  1. Assessing Writing Jul 2026
    LAWE-CL2: Multi-agent LLM-based automated writing evaluation system integrating linguistic features with fine-tuning for Chinese L2 writing assessment ↗
    Xuelin Wang; Qihao Yang; Yuxin Hao; Zhijun Wang; Sijia Guo
    assessment artificial intelligence multilingual writers
  2. Assessing Writing Jul 2026
    Investigating the impact of ChatGPT-assisted self-assessment on college students' writing development: Insights from diverse linguistic backgrounds ↗
    Hamidreza Moeiniasl
    assessment artificial intelligence
  3. Argumentation May 2026
    Do You Want to Build a Straw Man?: Evidentiary and Argumentative Modes of the Straw Man Fallacy ↗
    Eugen Octav Popa
    discourse analysis assessment
  4. Written Communication May 2026
    Beyond Co-Regulation: Interplay as a Methodological Framework for Examining Self-Regulation in Generative AI-Assisted Writing ↗
    Ryan Roderick; Susan Tanner
    writing pedagogy collaborative writing assessment empirical research artificial intelligence
  5. Journal of Response to Writing Apr 2026
    Examining Automated Writing Evaluation Error Coverage in Relation to Uptake and Retention ↗
    Mehrzad, Mohaddeseh; Rahimi, Mohammad; Link, Stephanie
    genre theory writing pedagogy teacher development revision assessment artificial intelligence multilingual writers grammar and mechanics