Written Communication Jan 2010

Linguistic Features of Writing Quality

Danielle S. McNamara University of Memphis ; Scott A. Crossley Mississippi State University ; Philip M. McCarthy University of Memphis

Abstract

In this study, a corpus of expert-graded essays, based on a standardized scoring rubric, is computationally evaluated so as to distinguish the differences between those essays that were rated as high and those rated as low. The automated tool, Coh-Metrix, is used to examine the degree to which high- and low-proficiency essays can be predicted by linguistic indices of cohesion (i.e., coreference and connectives), syntactic complexity (e.g., number of words before the main verb, sentence structure overlap), the diversity of words used by the writer, and characteristics of words (e.g., frequency, concreteness, imagability). The three most predictive indices of essay quality in this study were syntactic complexity (as measured by number of words before the main verb), lexical diversity (as measured by the Measure of Textual Lexical Diversity), and word frequency (as measured by Celex, logarithm for all words). Using 26 validated indices of cohesion from Coh-Metrix, none showed differences between high- and low-proficiency essays and no indices of cohesion correlated with essay ratings. These results indicate that the textual features that characterize good student writing are not aligned with those features that facilitate reading comprehension. Rather, essays judged to be of higher quality were more likely to contain linguistic features associated with text difficulty and sophisticated language.

Journal
Written Communication
Published
2010-01-01
DOI
10.1177/0741088309351547
CompPile
Search in CompPile ↗
Open Access
Closed
Topics
grammar and mechanics race and writing
Export
BibTeX RIS

Citation Context

Cited by in this index (42)

  1. Caltabellotta et al. (2026)
    Measuring vocabulary use in L2 English and L2 French writing: How methodologica…
    Assessing Writing
  2. Li et al. (2026)
    Assessing the effects of explicit coherence instruction on EFL students’ integr…
    Assessing Writing
  3. Zao et al. (2025)
    Comparing the Readability of English-Language CEO Statements in Chinese and Ame…
    IEEE Transactions on Professional Communication
  4. Huang et al. (2025)
    Assessing L2 writing formality using syntactic complexity indices: A fuzzy eval…
    Assessing Writing
  5. Potter et al. (2025)
    Assessing academic language in tenth grade essays using natural language processing
    Assessing Writing
Show all 42 →
  1. Zappoli et al. (2024)
    A Synchronic and Diachronic Study of Students’ Essays in Italian High Schools: …
    Written Communication
  2. Tate et al. (2024)
    Linguistic Features of Secondary School Writing: Can Natural Language Processin…
    Written Communication
  3. Emmanouil et al. (2024)
    Linguistic factors affecting L1 language evaluation in argumentative essays of …
    Assessing Writing
  4. Wei et al. (2024)
    Effects of task-based language teaching on functional adequacy in L2 writing
    Assessing Writing
  5. Effatpanah et al. (2024)
    A mixed Rasch model analysis of multiple profiles in L2 writing
    Assessing Writing
  6. Wang et al. (2023)
    Writing Quality Predictive Modeling: Integrating Register-Related Factors
    Written Communication
  7. Tabari et al. (2023)
    Exploring new insights into the role of cohesive devices in written academic genres
    Assessing Writing
  8. Qian (2023)
    Use of lexical features in high-stakes tests: Evidence from the perspectives of…
    Assessing Writing
  9. Yang et al. (2023)
    Predicting EFL expository writing quality with measures of lexical richness
    Assessing Writing
  10. Golparvar et al. (2022)
    Unpacking the contribution of linguistic features to graph writing quality: An …
    Assessing Writing
  11. Huang et al. (2022)
    The trajectory of syntactic complexity development in L1 Chinese narrative writ…
    Assessing Writing
  12. Steiss et al. (2022)
    Dimensions of text-based analytical writing of secondary students
    Assessing Writing
  13. Uzun (2021)
    Performance prediction strengths of noun and verb phrases in L2 writing: Compar…
    Assessing Writing
  14. Riemenschneider et al. (2021)
    Linguistic complexity in teachers' assessment of German essays in high stakes testing
    Assessing Writing
  15. Maamuujav (2021)
    Examining lexical features and academic vocabulary use in adolescent L2 student…
    Assessing Writing
  16. McDonough et al. (2021)
    Instructor Evaluation of Business Student Writing: Does Language Play a Role?
    Business and Professional Communication Quarterly
  17. Jo (2021)
    Exploring General Versus Academic English Proficiency as Predictors of Adolesce…
    Written Communication
  18. Barrot et al. (2021)
    Complexity, accuracy, and fluency as indices of college-level L2 writers’ proficiency
    Assessing Writing
  19. Kyle (2020)
    The relationship between features of source text use and integrated writing quality
    Assessing Writing
  20. Troia et al. (2019)
    Multidimensional Levels of Language Writing Measures in Grades Four to Six
    Written Communication
  21. Vögelin et al. (2019)
    The influence of lexical features on teacher judgements of ESL argumentative essays
    Assessing Writing
  22. Lim (2019)
    An investigation of the text features of discrepantly-scored ESL essays: A mixe…
    Assessing Writing
  23. Liu et al. (2018)
    Connecting writing assessment with critical thinking: An exploratory study of a…
    Assessing Writing
  24. Kim et al. (2018)
    Modeling second language writing quality: A structural equation investigation o…
    Assessing Writing
  25. Barkaoui et al. (2018)
    The effects of writing mode and computer ability on L2 test-takers' essay chara…
    Assessing Writing
  26. Balling (2018)
    No Effect of Writing Advice on Reading Comprehension
    Journal of Technical Writing and Communication
  27. Banerjee et al. (2015)
    Keeping up with the times: Revising and refreshing a rating scale
    Assessing Writing
  28. Aryadoust et al. (2015)
    Predicting EFL writing ability from levels of mental representation measured by…
    Assessing Writing
  29. Aull (2015)
    Connecting writing and language in assessment: Examining style, tone, and argum…
    Assessing Writing
  30. McNamara et al. (2015)
    A hierarchical classification approach to automated essay scoring
    Assessing Writing
  31. Roscoe et al. (2014)
    The Writing Pal Intelligent Tutoring System: Usability Testing and Development
    Computers and Composition
  32. Crossley et al. (2014)
    What Is Successful Writing? An Investigation Into the Multiple Ways Writers Can…
    Written Communication
  33. Struthers et al. (2013)
    Assessing cohesion in children’s writing: Development of a checklist
    Assessing Writing
  34. Dryer (2013)
    Scaling Writing Ability: A Corpus-Driven Inquiry
    Written Communication
  35. Uccelli et al. (2013)
    Mastering Academic Language: Organization and Stance in the Persuasive Writing …
    Written Communication
  36. Thonney (2012)
    “That's Beyond the Scope of This Paper”: Analyzing the Functions of a Familiar …
    Rhetoric Review
  37. Crossley et al. (2011)
    The Development of Writing Proficiency as a Function of Grade Level: A Linguist…
    Written Communication

References (59) · 3 in this index

  1. Baayen, R.H. (1995)
    The CELEX lexical database
  2. 10.1075/sl.7.3.05bam
     ↗
  3. Biber, D. ( 2003). Variation among university spoken and written registers: A new multidimensional analysis. …
     ↗
  4. Bormuth, J.R. (1969)
    Development of readability analyses
  5. Brace, N. (2006)
    SPSS for psychologists: A guide to data analysis using SPSS for Windows
Show all 59 →
  1. Chall, J. (1995)
    Readability revisited: The new Dale-Chall readability formula
  2. Charniak, E.
    Proceedings of the First Conference on North American Chapter of the Association for Comp…
  3. Collins (1998)
    Strategies for Struggling Writers
    College Composition and Communication
  4. 10.1080/14640748108400805
     ↗
  5. 10.1080/08351818409389208
     ↗
  6. 10.1111/j.1540-4781.2007.00507.x
     ↗
  7. 10.2307/1495373
     ↗
  8. 10.2307/747483
     ↗
  9. DeVillez, R. (2003)
    Writing: Step by step
  10. 10.1017/CBO9780511816796.038
     ↗
  11. 10.4324/9780203506929
     ↗
  12. Field, A. (2005)
    Discovering statistics using SPSS
  13. 10.4324/9780203463932_UC_and_the_SAT
     ↗
  14. 10.21236/ADA221854
     ↗
  15. 10.1075/z.74
     ↗
  16. 10.1075/idj.15.3.02gra
     ↗
  17. Graesser, A.C., McNamara, D.S. & Louwerse, M.M. (2003). What do readers need to learn in order to process coh…
  18. 10.3758/BF03195564
     ↗
  19. Haberlandt, K. (1985)
    Journal of Experimental Psychology: Human Learning and Memory
  20. Hall, C. (2007)
    University of Arizona Working Papers in Linguistics
  21. Halliday, M.A.K. (1976)
    Cohesion in English
  22. Just, M.A. (1987)
    The psychology of reading and language comprehension
  23. 10.1037/0033-295X.99.1.122
     ↗
  24. Kellogg (2008)
    Training writing skills: A cognitive developmental perspective
    Journal of Writing Research
  25. 10.4324/9780203936399
     ↗
  26. Leki, I. ( 1993). Reciprocal themes in ESL reading and writing. In J. Carson & I. Leki (Eds.), Reading in the…
  27. Light, R. (2001)
    Making the most of college
     ↗
  28. Lightman, E.J., McCarthy, P.M., Dufty, D.F. & McNamara, D.S. (2007). The structural organization of high scho…
  29. 10.1016/j.system.2005.02.002
     ↗
  30. Louwerse, M.M. (2001)
    10.1515/cogl.2002.005
    Cognitive Linguistics  ↗
  31. Louwerse, M.M., McCarthy, P.M., McNamara, D.S. & Graesser, A.C. (2004). Variation in language and cohesion ac…
  32. 10.2190/1LN8-7BQE-8TN0-M91L
     ↗
  33. McCarthy, P.M. (2005)
    Dissertation Abstracts International
  34. McCarthy, P.M., Briner, S.W., Rus, V. & McNamara, D.S. (2007). Textual signatures: Identifying text-types usi…
     ↗
  35. 10.1177/0265532207080767
     ↗
  36. McCarthy, P.M.
    Proceedings of the 19th Annual Florida Artificial Intelligence Research Society Internati…
  37. 10.1207/S15326985EP3501_3
     ↗
  38. 10.1037/h0087352
     ↗
  39. 10.4324/9780203810033
     ↗
  40. McNamara, D.S.
    Discourse Processes
  41. 10.3758/BF03195736
     ↗
  42. 10.1093/ijl/3.4.235
     ↗
  43. National Center for Educational Statistics. (2003)
    2002 NAEP Writing 2002 State Snapshot Records (NCES 2003-532)
  44. 10.1080/01638530709336895
     ↗
  45. 10.3102/0002831206298171
     ↗
  46. 10.1207/s1532690xci0102_1
     ↗
  47. 10.1037/0278-7393.31.6.1281
     ↗
  48. Raimes, A. ( 2001). What unskilled ESL students do as they write: A classroom study of composing. In T. Silva…
  49. Ransdell, S. (2003)
    Arob@se
  50. Raynor, A. (1994)
    The psychology of reading
  51. 10.1006/jecp.1996.0054
     ↗
  52. Tabachnick, B.G. (1989)
    Using multivariate statistics
  53. Todd et al. (2007)
    Coherence, cohesion and comments on students’ academic essays
    Assessing Writing
  54. 10.2307/356693
     ↗
CrossRef global citation count: 342 View in citation network → Build reading path →

Related Articles

  1. Res Rhetorica Jan 2026
    Review/Recenzja: Nancy Organ. 2024. Data Visualization for People of All Ages. Oxon: CRC Press; and Jen Christiansen. 2023. Building Science Graphics: An Illustrated Guide to Communicating Science Through Diagrams and Visualizations. Oxon: CRC Press ↗
    Ewa Modrzejewska
    modern rhetorical theory rhetorical criticism discourse analysis digital rhetoric multimodality grammar and mechanics race and writing book reviews
  2. Research in the Teaching of English Aug 2025
    Useful and Appropriate: Preservice ELA Teacher Reactions to Feedback on EL Student Writing ↗
    Meghan Odsliv Bratkovich; Michael B. Sherry; Homood Alharbi
    teacher development revision qualitative research multilingual writers translingualism grammar and mechanics race and writing
  3. Journal of Academic Writing Apr 2025
    Corpus Integration in L2 Discipline-Specific Writing Courses ↗
    Karla Csürös; Claudiu Gherasim; Mădălina Chitez
    writing pedagogy writing across the curriculum revision grammar and mechanics race and writing
  4. Technical Communication Quarterly Oct 2024
    Editors’ Use of Comprehensive Style Guides: The Case of Singular They ↗
    Jo Mackiewicz; Shaya Kraut; Allison Durazzi
    rhetorical criticism discourse analysis first-year composition writing pedagogy graduate education teacher development writing centers technical communication professional writing digital rhetoric multilingual writers grammar and mechanics literacy studies race and writing public rhetoric editorial matter
  5. Pedagogy Jan 2024
    The Radical Role of Student Writing in Composition ↗
    Jessica Masterson
    modern rhetorical theory rhetorical criticism genre theory discourse analysis cultural rhetorics first-year composition writing pedagogy graduate education teacher development argument collaborative writing professional writing grammar and mechanics literacy studies race and writing gender and writing