Annotating Argument Schemes

Jacky Visser University of Dundee ; John Lawrence University of Dundee ; Chris Reed University of Dundee ; Jean Wagemans ; Douglas Walton University of Windsor

Abstract

AbstractArgument schemes are abstractions substantiating the inferential connection between premise(s) and conclusion in argumentative communication. Identifying such conventional patterns of reasoning is essential to the interpretation and evaluation of argumentation. Whether studying argumentation from a theory-driven or data-driven perspective, insight into the actual use of argumentation in communicative practice is essential. Large and reliably annotated corpora of argumentative discourse to quantitatively provide such insight are few and far between. This is all the more true for argument scheme corpora, which tend to suffer from a combination of limited size, poor validation, and the use of ad hoc restricted typologies. In the current paper, we describe the annotation of schemes on the basis of two distinct classifications: Walton’s taxonomy of argument schemes, and Wagemans’ Periodic Table of Arguments. We describe the annotation procedure for each, and the quantitative characteristics of the resulting annotated text corpora. In doing so, we extend the annotation of the preexisting US2016 corpus of televised election debates, resulting in, to the best of our knowledge, the two largest consistently annotated corpora of schemes in argumentative dialogue publicly available. Based on evaluation in terms of inter-annotator agreement, we propose further improvements to the guidelines for annotating schemes: the argument scheme key, and the Argument Type Identification Procedure.

Journal
Argumentation
Published
2021-03-01
DOI
10.1007/s10503-020-09519-x
CompPile
Open Access
OA PDF Hybrid
Topics
Export

Citation Context

References (86) · 3 in this index

  1. Al Khatib, K., H. Wachsmuth, J. Kiesel, M. Hagen, and B. Stein. 2016. A news editorial corpus for mining argu…
  2. Argumentation
  3. Atkinson, K., and T. Bench-Capon. 2018. Taking account of the actions of others in value-based reasoning. Art…
    Artificial Intelligence  
  4. How to Do Things with Words
  5. Handbook of Formal Argumentation
Show all 86 →
  1. Bilu, Y., D. Hershcovich, and N. Slonim. 2015. Automatic claim negation: Why, how and when. In Proceedings of…
  2. Computational Logic in Multi-Agent Systems
  3. Chesñevar, C., S. Modgil, I. Rahwan, C. Reed, G. Simari, M. South, G. Vreeswijk, S. Willmott, et al. 2006. To…
    The Knowledge Engineering Review  
  4. Cohen, J. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20(1…
    Educational and Psychological Measurement  
  5. Argumentation in Science Education
  6. Dusmanu, M., E. Cabrio, and S. Villata. 2017. Argument mining on twitter: Arguments, facts and sources. In Pr…
  7. Duthie, R., J. Lawrence, K. Budzynska, and C. Reed. 2016. The CASS technique for evaluating the performance o…
  8. Encyclopedia of Language & Linguistics
  9. Feng, V.W., and G. Hirst. 2011. Classifying arguments by scheme. In Proceedings of the 49th Annual Meeting of…
  10. Fleiss, J. 1971. Measuring nominal scale agreement among many raters. Psychological Bulletin 76 (5): 378–382.
    Psychological Bulletin  
  11. Argumentation
  12. Crucial Concepts in Argumentation Theory
  13. Ghosh, D., S. Muresan, N. Wacholder, M. Aakhus, and M. Mitsui. 2014. Analyzing argumentative discourse units …
  14. Actes 13émes Journées d’Intelligence Artificielle Fondamentale (JIAF 2019)
  15. Proceedings of the 9th Conference of the International Society for the Study of Argumentation
  16. Proceedings of the 2nd Workshop on Advances in Argumentation in Artificial Intelligence (…
  17. Green, N. 2015. Identifying argumentation schemes in genetics research articles. In Proceedings of the 2nd Wo…
  18. Hansen, H.V., and D. Walton. 2013. Argument kinds and argument roles in the ontario provincial election. Jour…
    Journal of Argumentation in Context  
  19. Hassan, N., C. Li, and M. Tremayne. 2015. Detecting check-worthy factual claims in presidential debates. In P…
  20. Hastings, A.C. 1963. A reformulation of the modes of reasoning in argumentation. Ph.D. thesis, Northwestern U…
  21. Proceedings of the 5th Conference of the International Society for the Study of Argumentation
  22. Jaradat, I., P. Gencheva, A. Barrón-Cedeño, L. Màrquez, and P. Nakov. 2018. Claimrank: Detecting check-worthy…
  23. Argumentation
  24. Kienpointner, M. 1992. Alltagslogik. Struktur and Funktion von Argumentationsmustern [Everyday Logic. Structu…
  25. Kokciyan, N., I. Sassoon, A. Young, M. Chapman, T. Porat, M. Ashworth, V. Curcin, S. Modgil, S. Parsons, and …
  26. Televised Presidential Debates and Public Policy Communication and Society
  27. Landis, J., and G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics 3: 159–174.
    Biometrics  
  28. Proceedings of the 6th International Conference on Computational Models of Argument (COMM…
  29. Lawrence, J., and C. Reed. 2019. Argument mining: A survey. Computational Linguistics. https://doi.org/10.116…
    Computational Linguistics  
  30. Lawrence, J., R. Duthie, K. Budzynska, and C. Reed. 2016. Argument analytics. In The 6th International Confer…
  31. Lawrence, J., J. Visser, and C. Reed. 2019. An online annotation assistant for argument schemes. In Proceedin…
  32. Lindahl, A., L. Borin, and J. Rouces. 2019. Towards assessing argumentation annotation—A first step. In Proce…
  33. Miller, T., M. Sukhareva, and I. Gurevych. 2019. A streamlined method for sourcing discourse-level argumentat…
  34. Musi, E., D. Ghosh, and S. Muresan. 2016. Towards feasible guidelines for the annotation of argument schemes.…
  35. Naderi, N., and G. Hirst. 2018. Automated fact-checking of claims in argumentative parliamentary debates. In …
  36. Biological Identification
  37. Park, J., and C. Cardie. 2014. Identifying appropriate support for propositions in online user comments. In P…
  38. Park, J., and C. Cardie. 2018. A corpus of erulemaking user comments for measuring evaluability of arguments.…
  39. Patwari, A., D. Goldwasser, and S. Bagchi. 2017. Tathya: A multi-classifier system for detecting check-worthy…
  40. Peldszus, A., and M. Stede. 2013. From argument diagrams to argumentation mining in texts: A survey. Internat…
    International Journal of Cognitive Informatics and Natural Intelligence (IJCINI)  
  41. The New Rhetoric: A Treatise on Argumentation
  42. Peters, G., and J.T. Woolley. 1999. The American Presidency Project. http://www.presidency.ucsb.edu. Accessed…
  43. Peters, G., and J.T. Woolley. 2016. Presidential debate at Hofstra University in Hempstead, New York, Septemb…
  44. Prasad, R., N. Dinesh, A. Lee, E. Miltsakaki, L. Robaldo, A. Joshi, and B. Webber. 2008. The Penn discourse T…
  45. Argumentation in Artificial Intelligence
  46. Linguistics in the Twenty-First Century
  47. Proceedings of the 7th Conference of the International Society for the Study of Argumenta…
  48. The Persuasion Handbook: Developments in Theory and Practice
  49. Inference in Argumentation: A Topics-Based Approach to Argument Schemes
  50. Rinott, R., L. Dankin, C.A. Perez, M.M. Khapra, E. Aharoni, and N. Slonim. 2015. Show me your evidence-an aut…
  51. Ars Topica: The Classical Technique of Constructing Arguments from Aristotle to Cicero
  52. Redelijke argumenten. Een onderzoek naar normen voor kritische lezers [Reasonable Argumen…
  53. Speech Acts: An Essay in the Philosophy of Language
  54. Skeppstedt, M., A. Peldszus, and M. Stede. 2018. More or less controlled elicitation of argumentative text: E…
  55. Song, Y., M. Heilman, B. Beigman Klebanov, and P. Deane. 2014. Applying argumentation schemes for essay scori…
  56. Argumentation Mining
  57. Strategic Maneuvering in Argumentative Discourse: Extending the Pragma-dialectical Theory…
  58. Proceedings of the 9th Conference of the International Society for the Study of Argumenta…
  59. Argumentation, Communication, and Fallacies: A Pragma-Dialectical Perspective
  60. Argumentatietheorie [Argumentation Theory]
  61. Argumentative Indicators in Discourse: A Pragma-Dialectical Study
  62. Argumentation and artificial intelligence. In Handbook of Argumentation Theory
  63. The 16th ArgDiaP Conference
  64. Proceedings of the 11th International Conference on Language Resources and Evaluation (LR…
  65. Proceedings of the 7th International Conference on Computational Models of Argument (COMM…
  66. Visser, J., B. Konat, R. Duthie, M. Koszowy, K. Budzynska, and C. Reed. 2019a. Argumentation in the 2016 US p…
    Language Resources and Evaluation  
  67. Visser, J., J. Lawrence, J. Wagemans, and C. Reed. 2019b. An annotated corpus of argument schemes in us elect…
  68. Voss, E.G. 1952. The history of keys and phylogenetic trees in systematic biology. Journal of the Science Lab…
    Journal of the Science Laboratories, Denison University
  69. Argumentation, Objectivity, and Bias: Proceedings of the 11th International Conference of…
  70. Wagemans, J.H.M. 2017. Periodic Table of Arguments: The atomic building blocks of persuasive discourse. http:…
  71. Wagemans, J.H.M. 2018a. Analogy, similarity, and the periodic table of arguments. Studies in Logic, Grammar a…
    Studies in Logic, Grammar and Rhetoric  
  72. Argumentation and Inference: Proceedings of the 2nd European Conference on Argumentation
  73. Wagemans, J.H.M. 2019a. Argument Type Identification Procedure (ATIP)—version 2. http://www.periodic-table-of…
  74. Wagemans, J.H.M. 2019b. Four basic argument forms. Research in Language 17(1): 57–69. https://doi.org/10.2478…
    Research in Language  
  75. Argumentation Schemes for Presumptive Reasoning
  76. Fundamentals of Critical Argumentation
  77. Walton, D. 2011. Argument mining by applying argumentation schemes. Studies in Logic 4(1): 38–64.
    Studies in Logic
  78. Walton, D. 2012. Using argumentation schemes for argument extraction: A bottom-up method. International Journ…
    International Journal of Cognitive Informatics and Natural Intelligence  
  79. Walton, D., and F. Macagno. 2015. A classification system for argumentation schemes. Argument and Computation…
    Argument and Computation  
  80. Argumentation Schemes
  81. Wyner, A., J. Schneider, K. Atkinson, and T. Bench-Capon. 2012. Semi-automated argumentative analysis of onli…
    Proceedings of COMMA