Annotating Argument Schemes

Jacky Visser University of Dundee ; John Lawrence University of Dundee ; Chris Reed University of Dundee ; Jean Wagemans ; Douglas Walton University of Windsor

Abstract

AbstractArgument schemes are abstractions substantiating the inferential connection between premise(s) and conclusion in argumentative communication. Identifying such conventional patterns of reasoning is essential to the interpretation and evaluation of argumentation. Whether studying argumentation from a theory-driven or data-driven perspective, insight into the actual use of argumentation in communicative practice is essential. Large and reliably annotated corpora of argumentative discourse to quantitatively provide such insight are few and far between. This is all the more true for argument scheme corpora, which tend to suffer from a combination of limited size, poor validation, and the use of ad hoc restricted typologies. In the current paper, we describe the annotation of schemes on the basis of two distinct classifications: Walton’s taxonomy of argument schemes, and Wagemans’ Periodic Table of Arguments. We describe the annotation procedure for each, and the quantitative characteristics of the resulting annotated text corpora. In doing so, we extend the annotation of the preexisting US2016 corpus of televised election debates, resulting in, to the best of our knowledge, the two largest consistently annotated corpora of schemes in argumentative dialogue publicly available. Based on evaluation in terms of inter-annotator agreement, we propose further improvements to the guidelines for annotating schemes: the argument scheme key, and the Argument Type Identification Procedure.

Journal
Argumentation
Published
2021-03-01
DOI
10.1007/s10503-020-09519-x
CompPile
Search in CompPile ↗
Open Access
OA PDF Hybrid
Topics
Export

Citation Context

Cites in this index (3)

  1. Argumentation
  2. Argumentation
  3. Argumentation
Also cites 43 works outside this index ↓
  1. Atkinson, K., and T. Bench-Capon. 2018. Taking account of the actions of others in value-based reasoning. Art…
    Artificial Intelligence  
  2. Bilu, Y., D. Hershcovich, and N. Slonim. 2015. Automatic claim negation: Why, how and when. In Proceedings of…
  3. Chesñevar, C., S. Modgil, I. Rahwan, C. Reed, G. Simari, M. South, G. Vreeswijk, S. Willmott, et al. 2006. To…
    The Knowledge Engineering Review  
  4. Cohen, J. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20(1…
    Educational and Psychological Measurement  
  5. Argumentation in Science Education
  6. Dusmanu, M., E. Cabrio, and S. Villata. 2017. Argument mining on twitter: Arguments, facts and sources. In Pr…
  7. Duthie, R., J. Lawrence, K. Budzynska, and C. Reed. 2016. The CASS technique for evaluating the performance o…
  8. Encyclopedia of Language & Linguistics
  9. Fleiss, J. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin 76 (5): 378–382.
    Psychological Bulletin  
  10. Ghosh, D., S. Muresan, N. Wacholder, M. Aakhus, and M. Mitsui. 2014. Analyzing argumentative discourse units …
  11. Green, N. 2015. Identifying argumentation schemes in genetics research articles. In Proceedings of the 2nd Wo…
  12. Hansen, H.V., and D. Walton. 2013. Argument kinds and argument roles in the ontario provincial election. Jour…
    Journal of Argumentation in Context  
  13. Hassan, N., C. Li, and M. Tremayne. 2015. Detecting check-worthy factual claims in presidential debates. In P…
  14. Jaradat, I., P. Gencheva, A. Barrón-Cedeño, L. Màrquez, and P. Nakov. 2018. Claimrank: Detecting check-worthy…
  15. Televised Presidential Debates and Public Policy Communication and Society
  16. Landis, J., and G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics 3: 159–174.
    Biometrics  
  17. Lawrence, J., and C. Reed. 2019. Argument mining: A survey. Computational Linguistics. https://doi.org/10.116…
    Computational Linguistics  
  18. Lawrence, J., J. Visser, and C. Reed. 2019. An online annotation assistant for argument schemes. In Proceedin…
  19. Lindahl, A., L. Borin, and J. Rouces. 2019. Towards assessing argumentation annotation—A first step. In Proce…
  20. Miller, T., M. Sukhareva, and I. Gurevych. 2019. A streamlined method for sourcing discourse-level argumentat…
  21. Musi, E., D. Ghosh, and S. Muresan. 2016. Towards feasible guidelines for the annotation of argument schemes.…
  22. Naderi, N., and G. Hirst. 2018. Automated fact-checking of claims in argumentative parliamentary debates. In …
  23. Park, J., and C. Cardie. 2014. Identifying appropriate support for propositions in online user comments. In P…
  24. Patwari, A., D. Goldwasser, and S. Bagchi. 2017. Tathya: A multi-classifier system for detecting check-worthy…
  25. Peldszus, A., and M. Stede. 2013. From argument diagrams to argumentation mining in texts: A survey. Internat…
    International Journal of Cognitive Informatics and Natural Intelligence (IJCINI)  
  26. The Persuasion Handbook: Developments in Theory and Practice
  27. Inference in Argumentation: A Topics-Based Approach to Argument Schemes
  28. Rinott, R., L. Dankin, C.A. Perez, M.M. Khapra, E. Aharoni, and N. Slonim. 2015. Show me your evidence-an aut…
  29. Ars Topica: The Classical Technique of Constructing Arguments from Aristotle to Cicero
  30. Speech Acts: An Essay in the Philosophy of Language
  31. Skeppstedt, M., A. Peldszus, and M. Stede. 2018. More or less controlled elicitation of argumentative text: E…
  32. Song, Y., M. Heilman, B. Beigman Klebanov, and P. Deane. 2014. Applying argumentation schemes for essay scori…
  33. Argumentation Mining
  34. Strategic Maneuvering in Argumentative Discourse: Extending the Pragma-dialectical Theory…
  35. Argumentative Indicators in Discourse: A Pragma-Dialectical Study
  36. Argumentation and artificial intelligence. In Handbook of Argumentation Theory
  37. Proceedings of the 7th International Conference on Computational Models of Argument (COMM…
  38. Visser, J., B. Konat, R. Duthie, M. Koszowy, K. Budzynska, and C. Reed. 2019a. Argumentation in the 2016 US p…
    Language Resources and Evaluation  
  39. Wagemans, J.H.M. 2018a. Analogy, similarity, and the periodic table of arguments. Studies in Logic, Grammar a…
    Studies in Logic, Grammar and Rhetoric  
  40. Wagemans, J.H.M. 2019b. Four basic argument forms. Research in Language 17(1): 57–69. https://doi.org/10.2478…
    Research in Language  
  41. Walton, D. 2012. Using argumentation schemes for argument extraction: A bottom-up method. International Journ…
    International Journal of Cognitive Informatics and Natural Intelligence  
  42. Walton, D., and F. Macagno. 2015. A classification system for argumentation schemes. Argument and Computation…
    Argument and Computation  
  43. Argumentation Schemes