Assessing Writing Apr 2026 Open Access

How do L2 writing subskills interact hierarchically? Insights from diagnostic classification models

Farshad Effatpanah TU Dortmund University ; Hamdollah Ravand Vali Asr University of Rafsanjan ; Mahmoud Abdi Tabari University of Nevada, Reno ; Yi-Hsin Chen ; Olga Kunina-Habenicht TU Dortmund University

Abstract

This study examined the hierarchical structure among second/foreign language (L2) writing subskills using a Hierarchical Diagnostic Classification Model (HDCM). A pool of 500 essays composed by English as a Foreign Language (EFL) students was assessed by four experienced EFL teachers using the Empirically-derived Descriptor-based Diagnostic (EDD) checklist. Based on a literature review and the expertise of three content experts, several models were developed to reflect various hierarchical interactions among L2 writing subskills, including linear, divergent, convergent, independent, unstructured, mixed, and higher-order. The comparison of the models showed the presence of an unstructured interaction among L2 writing subskills, indicating that content is the foundational subskill for the mastery of vocabulary, grammar, organization, and mechanics. Higher mastery classes were also associated with higher educational levels, greater frequency of English use, and longer exposure to L2. Understanding the hierarchical relationships among L2 writing subskills can improve targeted instructional strategies and assessment practices. • A constrained version of existing DCMs is represented by hierarchical DCMs. • Models were developed to show hierarchical interactions among L2 writing subskills. • An unstructured interaction among L2 writing subskills was identified. • Higher mastery classes were associated with higher educational levels. • The classes were associated with greater English use and longer L2 exposure.

Journal
Assessing Writing
Published
2026-04-01
DOI
10.1016/j.asw.2026.101029
CompPile
Search in CompPile ↗
Open Access
OA PDF Hybrid
Topics
assessment multilingual writers grammar and mechanics
Export
BibTeX RIS

Citation Context

Cited by in this index (0)

No articles in this index cite this work.

References (111) · 8 in this index

  1. Abdi Tabari (2024)
    The effects of task complexity on linguistic complexity and perception in English native …
    Language Awareness  ↗
  2. Tabari et al. (2022)
    Assessing linguistic complexity features in L2 writing: Understanding effects o…
    Assessing Writing
  3. Akaike (1974)
    A new look at the statistical model identification
    IEEE Transactions on Automatic Control  ↗
  4. Alderson (2005)
    Diagnosing foreign language proficiency: The interface between learning and assessment
  5. Bachman (2010)
    Language assessment in practice: Developing language assessments and justifying their use in the real world
Show all 111 →
  1. Bae (2010)
    An investigation of four writing traits and two tasks across two languages
    Language Testing  ↗
  2. Bereiter (1987)
    The psychology of written composition
  3. Birenbaum (1993)
    Diagnosing knowledge states in Algebra using the Rule-space model
    Journal for Research in Mathematics Education  ↗
  4. Bond (2020)
    Applying the Rasch model: Fundamental measurement in the human sciences
  5. Chapelle (1993)
    Communicative language proficiency: Definitions and implications for TOEFL 2000
  6. Chen (2013)
    Relative and absolute fit evaluation in cognitive diagnosis modeling
    Journal of Educational Measurement  ↗
  7. Chen (2023)
    Bayesian estimation of attribute hierarchy for cognitive diagnosis models
    Journal of Educational and Behavioral Statistics  ↗
  8. CHENOWETH et al. (2001)
    Fluency in Writing: Generating Text in L1 and L2
    Written Communication
  9. Chenoweth et al. (2003)
    The Inner Voice in Writing
    Written Communication
  10. Cumming (1998)
    Theoretical perspectives on writing
    Annual Review of Applied Linguistics  ↗
  11. Cumming (2006)
    Goals for academic writing: ESL students and their instructors
  12. de la Torre (2011)
    The generalized DINA model framework
    Psychometrika  ↗
  13. de la Torre (2016)
    A general method of empirical Q-matrix validation
    Psychometrika  ↗
  14. de la Torre (2018)
    Analysis of clinical data from cognitive diagnosis modeling framework
    Measurement and Evaluation in Counseling and Development  ↗
  15. Dong (2021)
    An optimal choice of cognitive diagnostic model for second language listening comprehensi…
    Frontiers in Psychology  ↗
  16. Ebel (1991)
    Essentials of educational measurement
  17. Eckes (2015)
    Introduction to many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments (2nd revised and updated edition)
  18. Effatpanah (2022)
    Exploring rater quality in rater-mediated assessment using the non-parametric item charac…
    Psychological Test and Assessment Modeling
  19. Effatpanah (2019)
    Diagnosing EFL learners’ writing ability: Adiagnostic classification modeling analysis
    Language Testing in Asia
  20. Effatpanah et al. (2024)
    A mixed Rasch model analysis of multiple profiles in L2 writing
    Assessing Writing
  21. Effatpanah (2026)
    Optimizing large-scale mathematical assessments: Leveraging hierarchical attribute struct…
    Educational Measurement: Issues and Practice
  22. Ellis (2005)
    Principles of instructed language learning
    System  ↗
  23. Flower et al. (1981)
    A Cognitive Process Theory of Writing
    College Composition and Communication
  24. Galbraith (2009)
    Writing as discovery
  25. Grabe (1996)
    Theory and practice of writing: An applied linguistic perspective
  26. Haertel (1989)
    Using restricted latent class models to map the skill structure of achievement items
    Journal of Educational Measurement  ↗
  27. Hartz, S. M. (2002). A Bayesian Framework for the unified Model for Assessing Cognitive abilities: Blending T…
  28. Hayes (1996)
    A new framework for understanding cognition and affect in writing
    The science of writing: Theories, methods, individual differences, and applications
  29. He et al. (2021)
    Diagnosing writing ability using China’s Standards of English Language Ability:…
    Assessing Writing
  30. Henson (2009)
    Defining a family of cognitive diagnosis models using log-linear models with latent variables
    Psychometrika  ↗
  31. Houts (2013)
    flexMIRTR user’s manual version 2: Flexible multilevel Multidimensional item analysis and test scoring
  32. Hu (1999)
    Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria v…
    Structural Equation Modeling: A Multidisciplinary Journal  ↗
  33. Huang et al. (2022)
    Individualized feedback to raters in language assessment: Impacts on rater effects
    Assessing Writing
  34. Huang (2022)
    Facilitating L2 writers’ metacognitive strategy use in argumentative writing using a proc…
    Frontiers in Psychology  ↗
  35. Johnson (2018)
    Measures of agreement to assess attribute-level classification accuracy and consistency f…
    Journal of Educational Measurement  ↗
  36. Junker (2001)
    Cognitive assessment models with few assumptions, and connections with nonparametric item…
    Applied Psychological Measurement  ↗
  37. Karelitz (2008)
    How binary skills obscure the transition from non-mastery to mastery
    Measurement: Interdisciplinary Research and Perspectives
  38. Kellogg (1996)
    A model of working memory in writing
    The science of writing: Theories, methods, individual differences, and applications
  39. Kessler (2022)
    The effects of topic familiarity on text quality, complexity, accuracy, and fluency: A co…
    TESOL Quarterly  ↗
  40. Khamboonruang (2025)
    Investigating the applicability of a diagnostic classification model in a Thai EFL classr…
    Language Testing in Asia  ↗
  41. Kim, Y.-H.. (2010). An argument-based validity inquiry into the Empirically-derived Descriptor-based Diagnost…
  42. Kim (2011)
    Diagnosing EAP writing ability using the reduced reparameterized unified model
    Language Testing  ↗
  43. Kim (2023)
    Linguistic alignment in second language acquisition: A methodological review
    System  ↗
  44. Kormos (2011)
    Task complexity and linguistic and discourse features of narrative writing performance
    Journal of Second Language Writing  ↗
  45. Krashen (1985)
    The input hypothesis: Issues and implications
  46. Kwon (2024)
    Modeling hierarchical attribute structures in diagnostic classification models with multi…
    Journal of Educational Measurement  ↗
  47. Lan (2022)
    Revisiting grammatical complexity in L2 writing via exploratory factor analysis
    Frontiers in Psychology  ↗
  48. Laufer (1995)
    Vocabulary size and use: Lexical richness in L2 written production
    Applied Linguistics  ↗
  49. Lee (2023)
    Diagnosing Chinese college-level English as a Foreign Language (EFL) learners’ …
    Assessing Writing
  50. Lee (2021)
    What linguistic features distinguish and predict L2 writing quality? A study of examinati…
    System  ↗
  51. Leighton (2007)
    Cognitive diagnostic assessment for education: Theory and applications
  52. Leighton (2004)
    The attribute hierarchy method for cognitive assessment: A variation on Tatsuoka’s rule-s…
    Journal of Educational Measurement  ↗
  53. Levelt (1989)
    Speaking: From intention to articulation. Cambridge, MA
  54. Li (2015)
    The selection of cognitive diagnostic models for a reading comprehension test
    Language Testing  ↗
  55. Li (2020)
    Cognitive diagnostic models for rater effects
    Frontiers in Psychology
  56. Linacre (1989)
    Many-facet Rasch measurement
  57. Linacre (2014)
    Facets Rasch measurement [Computer program]
  58. Linacre, J. M. (2024). A user’s guide to FACETS Rasch-model computer programs. Retrieved from: https://www.wi…
  59. Liu (2016)
    An application of M2 statistic to evaluate the fit of cognitive diagnostic models
    Journal of Educational and Behavioral Statistics  ↗
  60. Ma (2019)
    Cognitive diagnosis modeling using the GDINA R package
    Handbook of diagnostic classification models
  61. Ma, W., de la Torre, J., Sorrel, M., & Jiang, Zh (2025). GDINA: The generalized DINA model framework. R packa…
  62. Ma (2016)
    A sequential cognitive diagnosis model for polytomous responses
    The British Journal of Mathematical and Statistical Psychology  ↗
  63. Ma (2023)
    Learning latent and hierarchical structures in cognitive diagnosis models
    Psychometrika  ↗
  64. Maris (1999)
    Estimating multiple classification latent class models
    Psychometrika  ↗
  65. Maydeu-Olivares (2013)
    Goodness-of-fit assessment of item response theory models
    Measurement: Interdisciplinary Research and Perspectives
  66. Maydeu-Olivares (2014)
    Assessing approximate fit in categorical data analysis
    Multivariate Behavioral Research  ↗
  67. Messick (1989)
    Validity
    Educational measurement
  68. Milanovic (1996)
    A study of the decision-making behavior of composition markers
    Studies in language testing 3: Performance testing, cognition, and assessment
  69. Nation (2013)
    Learning vocabulary in another language
  70. Olive (2004)
    Working memory in writing: Empirical evidence from the dual-task Technique
    European Psychologist  ↗
  71. Olsen (2023)
    Improving students’ learning—the role of formative feedback: Experiences from a crash cou…
    Assessment Evaluation in Higher Education  ↗
  72. Pienemann (1998)
    Language processing and second language development: Processability theory
  73. Pu (2023)
    Language development for English-medium instruction: A longitudinal perspective on the us…
    Sustainability  ↗
  74. R Core Team (2025). R: A language and environment for statistical computing. R Foundation for Statistical Com…
  75. Rasch (1960)
    Probabilistic models for some intelligence and attainment tests (Expanded Ed.)
  76. Ravand (2019)
    Application of a hierarchical diagnostic classification model in assessing reading compre…
    Quantitative data analysis for language assessment Volume II: Advanced methods
  77. Ravand (2025)
    A didactic illustration of writing skill growth through a longitudinal diagnostic classif…
    Frontiers in Psychology  ↗
  78. Ravand (2024)
    Exploring interrelationships among L2 writing subskills: Insights from cognitive diagnost…
    Applied Measurement in Education  ↗
  79. Révész (2017)
    Investigating IELTS academic writing task 2: Relationships between cognitive writing proc…
    IELTS Research Reports: Online Series
  80. Rupp (2010)
    Diagnostic measurement: Theory, methods, and applications
  81. Sasaki (1996)
    Explanatory variables for EFL students’ expository writing
    Language Learning  ↗
  82. Schmidt (1990)
    The role of consciousness in second language learning
    Applied Linguistics  ↗
  83. Schoonen (2003)
    First language and second language writing: The role of linguistic knowledge, speed of pr…
    Language Learning  ↗
  84. Schoonen (2011)
    Modeling the development of L1 and EFL writing proficiency of secondary school students
    Language Learning  ↗
  85. Schwarz (1978)
    Estimating the dimension of a model
    Annals of Statistics  ↗
  86. Shahsavar (2019)
    Diagnosing English learners’ writing skills: A cognitive diagnostic modeling study
    Cogent Education  ↗
  87. Shi (2024)
    Diagnosing Chinese EFL learners’ writing ability using polytomous cognitive diagnostic models
    Language Testing  ↗
  88. Skehan (1998)
    A cognitive approach to language learning
  89. Stiggins (2002)
    Assessment crisis: The absence of assessment for learning
    Phi Delta Kappan  ↗
  90. Stoel (2006)
    On the likelihood ratio test in structural equation modeling when parameters are subject …
    Psychological Methods  ↗
  91. Tatsuoka (1983)
    Rule space: An approach for dealing with misconception based on item response theory
    Journal of Education Measurement  ↗
  92. Templin (2014)
    Hierarchical diagnostic classification models: A family of models for estimating and test…
    Psychometrika  ↗
  93. Templin (2006)
    Measurement of psychological disorders using cognitive diagnosis models
    Psychological Methods  ↗
  94. Thompson (2024)
    Improving instructional decision-making using diagnostic classification models
    Educational Measurement: Issues and Practice  ↗
  95. Tu (2019)
    Cognitive diagnostic models with attribute hierarchies: Model estimation with a restricte…
    Applied Psychological Measurement  ↗
  96. Tu (2017)
    A polytomous model of cognitive diagnostic assessment for graded data
    International Journal of Testing  ↗
  97. van Gelderen (2011)
    Does foreign language writing benefit from increased lexical fluency? Evidence from a cla…
    Language Learning  ↗
  98. Vasylets (2021)
    The effects of working memory and L2 proficiency on L2 writing
    Journal of Second Language Writing  ↗
  99. von Davier (2008)
    A general diagnostic model applied to language testing data
    British Journal of Mathematical and Statistical Psychology  ↗
  100. von Davier (2010)
    Hierarchical mixtures of diagnostic models
    Psychological Test and Assessment Modeling
  101. Weigle (2002)
    Assessing writing
  102. Xie (2016)
    Diagnosing university students’ academic writing in English: Is cognitive diagnostic mode…
    Educational Psychology  ↗
  103. Zhan (2020)
    A sequential higher order latent structural model for hierarchical attributes in cognitiv…
    Applied Psychological Measurement  ↗
  104. Zhang (2022)
    The relationship between lexical use and L2 writing quality: A case of two genres
    International Journal of Applied Linguistics  ↗
  105. Zhang (2020)
    On the sequential hierarchical cognitive diagnostic model
    Frontiers in Psychology  ↗
  106. Zheng (2012)
    Exploring long-term productive vocabulary development in an EFL context: The role of motivation
    System  ↗
CrossRef global citation count: 1 View in citation network → Build reading path →

Related Articles

  1. Assessing Writing Apr 2026
    Pursuing fair writing assessment: Halo effects in primary school foreign language writing in grade six ↗
    Ruth Trüb; Julian Lohmann; Jens Möller; Stefan D. Keller
    assessment multilingual writers grammar and mechanics
  2. Assessing Writing Jan 2026
    The effects of online resource use on L2 learners’ computer-mediated writing processes and written products ↗
    Honglan Wang; Jookyoung Jung
    revision argument assessment digital rhetoric multilingual writers grammar and mechanics affect and writing
  3. Assessing Writing Jan 2026
    How reliable and valid is peer evaluation in adolescents’ L2 argumentative writing? ↗
    Albert W. Li; Steve Graham
    writing pedagogy teacher development argument assessment multilingual writers grammar and mechanics literacy studies
  4. Assessing Writing Oct 2025
    Assessing L2 writing formality using syntactic complexity indices: A fuzzy evaluation approach ↗
    Zhiyun Huang; Guangyao Chen; Zhanhao Jiang
    assessment multilingual writers grammar and mechanics
  5. Assessing Writing Oct 2025
    Judgment accuracy in primary school EFL writing assessment: Do text characteristics matter? ↗
    Ruth Trüb; Jens Möller; Julian Lohmann; Thorben Jansen; Stefan D. Keller
    teacher development assessment multilingual writers grammar and mechanics