Assessing Writing Jan 2026 Open Access

Extracting interpretable writing traits from a large language model

Paul Deane Educational Testing Service ; Andrew Hoang Educational Testing Service

Abstract

Large language models (LLMs) are increasingly used to support automated writing evaluation (AWE), both for purposes of scoring and feedback. However, LLMs present challenges to interpretability, making it hard to evaluate the construct validity of scoring and feedback models. BIOT (best interpretable orthogonal transformations) is a new method of analysis that makes dimensions of an embedding interpretable by aligning them with external predictors. It was originally developed to improve the interpretability of multidimensional scaling models. However, This paper shows that BIOT can be used to align LLM embeddings with an interpretable writing trait model developed using multidimensional analysis of classical NLP features to measure latent dimensions of writing style and writing quality. This makes it possible to determine whether an AWE model built using an LLM is aligned with known (and construct-relevant) dimensions of textual variation, supporting construct validity. Specifically, we examine the alignment between the hidden layers of deBERTA, a small LLM that has been shown to be useful for a variety of natural language processing applications, and a writing trait model developed through factor analysis of classical features used in existing AWE models. Specific dimensions of transformed deBERTA layers are strongly correlated with these classical factors. When the transformation matrix derived using BIOT is applied to token vectors, it is also possible to visualize which tokens in the original text contributed to high or low scores on a specific dimension. • Large language models (LLMs) are increasingly used to support automated writing evaluate (AWE). • LLMs present challenges to interpretability, making it hard to evaluate construct validity of scoring and feedback models. • BIOT is a new interpretation method that aligns embedding dimensions with external predictors. • Specifically, BIOT can be used to align LLM embeddings with classical NLP measures of aspects of style and writing quality. • This demonstrates a general method to determine whether an LLM latently represents construct-relevant dimensions.

Journal
Assessing Writing
Published
2026-01-01
DOI
10.1016/j.asw.2025.101011
CompPile
Search in CompPile ↗
Open Access
OA PDF Hybrid
Topics
assessment artificial intelligence
Export
BibTeX RIS

Citation Context

Cited by in this index (0)

No articles in this index cite this work.

References (56) · 2 in this index

  1. American Educational Research Association (AERA), American Psychological Association (APA), & National Counci…
  2. Attali (2006)
    Automated essay scoring with e-rater® V. 2. The Journal of Technology
    Learning and Assessment
  3. Attali (2008)
    A developmental writing scale
    ETS RR-08-19 ETS Research Report Series
  4. Attali (2015)
    Automated Trait Scores for TOEFL® Writing Tasks
    ETS RR-15-15 ETS Research Report Series
  5. Attali (2015)
    Automated trait scores for" GRE"® writing tasks
    Research Report ETS RR-15-15 ETS Research Report Series
Show all 56 →
  1. Bhardwaj, A., Di, W., & Wei, J. (2018). Deep Learning Essentials: Your hands-on guide to the fundamentals of …
  2. Bibal (2021)
    BIOT: explaining multidimensional nonlinear MDS embeddings using the best interpretable o…
    Neurocomputing  ↗
  3. Biber (1988)
    Variation across Speech and Writing
  4. Biber (1989)
    A typology of English texts
    Linguistics  ↗
  5. Biber, D. (2006). University language: a corpus-based study of spoken and written registers. Benjamins.
     ↗
  6. Biber (2012)
    Register as a predictor of linguistic variation
    Corpus Linguistics and Linguistic Theory  ↗
  7. Biber (2019)
    Multi-dimensional analysis: A historical synopsis
    Multi-dimensional analysis: Research methods and current issues
  8. Boyd, R.L., Ashokkumar, A., Seraj, S., & Pennebaker, J.W. (2022). The development and psychometric properties…
  9. Bricken (2023)
    Towards Monosemanticity: Decomposing Language Models with Dictionary Learning
    AI Transformer Circuits Thread
  10. Burstein (2023)
    10.46999/VCAE5025
    The Duolingo English Test Responsible AI Standards  ↗
  11. Burstein (2004)
    Automated essay evaluation: The Criterion online writing service
    AI Magazine
  12. Burstein (1999)
    Automated essay scoring for nonnative English speakers
    In ASSESSEVALNLP '99: Proceedings of a Symposium on Computer Mediated Language Assessment and Evaluation in Natural Language Processing  ↗
  13. Chung (2018)
    What do we know when we LIWC a person? Text analysis as an assessment tool for traits, pe…
    The Sage Handbook of Personality and Individual Differences
  14. Crossley et al. (2022)
    The persuasive essays for rating, selecting, and understanding argumentative an…
    Assessing Writing
  15. Crossley (2023)
    A large-scale corpus for assessing written argumentation: PERSUADE 2.0
    Assessing Writing
  16. Crossley (2008)
    Assessing text readability using cognitively based indices
    TESOL Quarterly  ↗
  17. Deane (2006)
    Differences in text structure and its implications for assessment of struggling readers
    Scientific Studies of Reading  ↗
  18. Deane (2024)
    Modeling Writing Traits in a Formative Essay Corpus
    ETS Research Report Series
  19. Farra (2015)
    June). Scoring persuasive essays using opinions and their targets
    In Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications  ↗
  20. Fiacco (2023)
    Functional Components as a Paradigm for Neural Model Explainability (Doctoral dissertation, Carnegie Mellon University)
  21. Flor (2013)
    June). Lexical tightness and text complexity
    In Proceedings of the Workshop on Natural Language Processing for Improving Textual Accessibility
  22. Grzegorczyk (2019)
    Vector representations of text data in deep learning
    arXiv Preprint arXiv
  23. Guenole (2024)
    Pseudo Factor Analysis of Language Embedding Similarity Matrices: New Ways to Model Latent Constructs Preprint
  24. He (2020)
    Deberta: Decoding-enhanced bert with disentangled Attention arXiv Preprint
  25. Helberg et al. (2018)
    Teaching textual awareness with DocuScope: Using corpus-driven tools and reflec…
    Assessing Writing
  26. Ishizaki (2012)
    Computer-aided rhetorical analysis
    Applied Natural Language Processing: Identification, Investigation and Resolution
  27. Jeong (2022)
    SciDeBERTa: Learning DeBERTa for Science Technology Documents and Fine-Tuning Information…
    IEEE Access  ↗
  28. Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., de Las Casas, D., Bressand, F., Lengye…
  29. Kahn (2007)
    Measuring emotional expression with the Linguistic Inquiry and Word Count
    The American Journal of Psychology  ↗
  30. Kandru (2023)
    July). Tenzin-Gyatso at SemEval-2023 Task 4: Identifying Human Values behind Arguments Us…
    In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)  ↗
  31. Kaufer, D. (2023). The DocuScope project History, theory and future directions. In Kaufer, D. & Ishizaki, S.,…
     ↗
  32. Kaufer (2016)
    Corpus Methods and Textual Visualization to Enhance Learning in Core Writing Courses
    In EDM (Workshops)
  33. Klebanov, B.B., Stab, C., Burstein, J., Song, Y., Gyawali, B., & Gurevych, I. (2016, August). Argumentation: …
     ↗
  34. Kruskal (1964)
    Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis
    Psychometrika  ↗
  35. Kruskal (1978)
    Multidimensional scaling
  36. Marion (2019)
    BIR: A method for selecting the best interpretable multidimensional scaling rotation usin…
    Neurocomputing  ↗
  37. Martin (2022)
    July). KDDIE at SemEval-2022 Task 11: Using DeBERTa for Named Entity Recognition
    In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)  ↗
  38. McCaffrey (2021)
    Best practices for constructed-response scoring
  39. Minh (2022)
    Explainable artificial intelligence: a comprehensive review
    Artificial Intelligence Review  ↗
  40. Pennebaker (2015)
    Linguistic Inquiry and word count: LIWC 2015 Operator’States Manual
  41. Phillips (2021)
    Four principles of explainable artificial intelligence
    NISTIR 8312) US Department of Commerce, National Institute of Standards; Technology
  42. Rahali (2023)
    End-to-end transformer-based models in textual-based NLP
    AI  ↗
  43. Rudin (2019)
    Stop explaining black box machine learning models for high stakes decisions and use inter…
    Nature Machine Intelligence  ↗
  44. Sheehan (2017)
    Validating automated measures of text complexity
    Educational Measurement: Issues and Practice  ↗
  45. Singh (2024)
    Rethinking Interpretability in the Era of Large Language Models
    arXiv
  46. Templeton (2024)
    Scaling monosemanticity AI Transformer Circuits Thread
  47. Wang (2019)
    Language models with transformers
    arXiv Preprint arXiv
  48. Xia, F., Li, B., Weng, Y., He, S., Sun, B., Li, S., & Zhao, J. (2022, July). LingJing at SemEval-2022 Task 3:…
     ↗
  49. Yngve, V.H. (1961). The depth hypothesis. In Jakobson, R. (Ed.), Proceedings of Symposia in Applied Mathemati…
     ↗
  50. Young (2018)
    Recent trends in deep learning based natural language processing [Review Article
    IEEE Computational Intelligence Magazine  ↗
  51. Yun (2014)
    On the iteration complexity of cyclic coordinate gradient descent methods
    SIAM Journal on Optimization  ↗
CrossRef global citation count: 0 View in citation network → Build reading path →

Related Articles

  1. Assessing Writing Jul 2026
    LAWE-CL2: Multi-agent LLM-based automated writing evaluation system integrating linguistic features with fine-tuning for Chinese L2 writing assessment ↗
    Xuelin Wang; Qihao Yang; Yuxin Hao; Zhijun Wang; Sijia Guo
    assessment artificial intelligence multilingual writers
  2. Assessing Writing Jul 2026
    Investigating the impact of ChatGPT-assisted self-assessment on college students' writing development: Insights from diverse linguistic backgrounds ↗
    Hamidreza Moeiniasl
    assessment artificial intelligence
  3. Assessing Writing Apr 2026
    Assessing fairness in finetuned scoring models with demographically restricted training data ↗
    Langdon Holmes; Wesley Morris; Scott Crossley; Joon Suh Choi
    assessment artificial intelligence race and writing
  4. Assessing Writing Apr 2026
    Generative artificial intelligence for automated writing evaluation: A systematic review of trends, efficacy, and challenges ↗
    Shadi I. Abudalfa; Jessie S. Barrot
    assessment artificial intelligence book reviews
  5. Journal of Technical Writing and Communication Mar 2026
    Canon to Code: Rhetorical Rulemaking for Generative AI Content Audits and Governance ↗
    Miriam F. Williams
    classical rhetoric modern rhetorical theory rhetorical criticism assessment technical communication artificial intelligence