uva.nl
ILLC

Master of Logic / ILLC

Projects

2nd Semester 2021/22: Argument-checking: Computational analysis of natural arguments

Instructors

[This is a link to the Project's Presentation May 11.]

ECTS
6
Description

Logic has traditionally looked at arguments trying to distinguish valid forms and fallacies. To this end, logical analyses usually abstract away from the context in which the arguments have been put forward, reconstructing them so as to reveal their underlying reasoning.

 

Different from most formal logical approaches, ‘argument-checking’ takes a different starting point. First, descriptively, argument-checking is interested in identifying arguments in natural language, and especially in contexts where the ‘logical form’ is difficult to see at first sight (for instance on social media, websites, or any type of online platform). Second, argument checking develops an approach to analyzing argument that is step-wise and amenable to training individuals with different levels of education.

 

The activity of argument-checking requires a set of skills or competencies for interpreting persuasive discourse, whether that is a single persuasive message or a complete text aimed at convincing the reader to believe something or to do something. 

 

As a stepwise procedure, argument-checking consists of the following:

  1. The activity of ‘argument detection’ aims at answering the question as to which discourse elements count as argumentative. Subtasks are, e.g., finding out what the main claim is that the author of the discourse wants to convey to their audience and which arguments have been put forward in support of that claim.

  2. Presupposing argument detection, ‘argument mapping’ is aimed at finding out how the argumentative elements contained in the discourse hang together, thus creating a structured picture of its argumentative fabric. 

  3. The next step is ‘argument type identification’, in which the individual conclusion-premise pairs represented in the argument map are labeled in terms of the Periodic Table of Arguments (PTA) by means of determining the main characteristics of the argument they instantiate.

  4. Once it has become clear what types of argument are represented in the text or discussion, ‘argument assessment’ can take place by asking specific critical questions relevant to their evaluation and labeling negative outcomes with corresponding names of fallacies.

 

Applied especially in the context of online media, argument-checking may function as a collegial tool to improve on the quality of online information, rather than just as an external assessment of natural arguments. This raises interesting questions about ethics or arguments, and more generally about the ethical framework within which to develop argument checking, as a human annotation activity or as an automatized process.

From this approach, a number of questions may be raised, to be clustered around two main themes:

  1. The role of human annotation in the process of argument-checking;

  2. The prospects of automatizing the process or argument-checking in the form of a glass-box AI;

  3. The ethical stance and boundaries of argument-checking.

 

Students may work on any of the following topics, or propose others:

  • Argument mining & argument mapping: contribute to further developing protocols and procedures;

  • Argument type identification (PTA): contribute to better specify steps for human annotation or for automatizing argument-checking;

  • Argument-checking: develop critical questions and fallacy identification

  • Argument-checking: describe the stepwise procedure in a way that is suitable for automation (possible collaboration with a software architect);

  • Adversarial vs cooperative forms of argumentation: discuss and problematize various ethical stances with respect to argumentation.

Organisation

Students are given in advance some material about analyzing natural argument and computational argumentation. A first session is organized, together with student(s) and teacher(s) to discuss the material and to set up the work of the coming weeks, especially the specific sub-project that each student wants to work on. Students then work autonomously for 3 weeks, with feedback on-demand sessions. A final mini conference to present the work to the whole group and the teacher(s). Different arrangements are possible, after consultation with the teacher(s) and MoL director.

Prerequisites

There are no formal prerequisites other than acquaintance with argumentation theory, which supposedly any MoL student has. Students will be required to familiarize, at the beginning of the project, with ‘argument-checking’, via basic literature to be circulated in due course.

Assessment

Students have to prepare a text of about 5000 words (excluding references), in the form of an academic paper, a technical report, or any other relevant format to be agreed upon with the teacher(s).

References

Aberdein, A., & Cohen, D. H. (2016). Introduction: Virtues and Arguments. Topoi, 35(2), 339–343. https://doi.org/10.1007/s11245-016-9366-3

Brave, R., Russo, F., Wagemans, J.H.M. (2022). Argument-Checking: A Critical Pedagogy Approach to Digital Literacy. In F. Ciracì, G. Miglietta & C. Gatto (Eds.), AIUCD 2022 - Culture digitali. Intersezioni: filosofia, arti, media. Proceedings della 11a conferenza nazionale, Lecce, 2022 (pp. 245-248). Associazione per l’Informatica Umanistica e la Cultura Digitale. DOI:https://doi.org/10.6092/unibo/amsacta/6848

Dalgleish, A., Girard, P., & Davies, M. (2017). Critical Thinking, Bias and Feminist Philosophy: Building a Better Framework through Collaboration. Informal Logic, 37(4), 351–369. https://doi.org/10.22329/il.v37i4.4794

Gobbo, F., Benini, M., & Wagemans, J.H.M. (2019). Annotation with Adpositional Argumentation: Guidelines for building a Gold Standard Corpus of argumentative discourse. Intelligenza Artificiale, 13(2), 155-172.

Gobbo, F., Benini, M., & Wagemans, J.H.M. (2021). Complex arguments in Adpositional Argumentation (AdArg). In M. D'Agostino, F.A. D’Asaro & C. Larese (Eds.), Proceedings of the Workshop on Advances in Argumentation in Artificial Intelligence (AI^3 2021), co-located with the 19th International Conference of the Italian Association for Artificial Intelligence (AIxIA 2021), Milan, November 29, 2021. URL =  http://ceur-ws.org/Vol-3086/

Gobbo, F., Benini, M., & Wagemans, J.H.M. (2022). More than relata refero: Representing the various roles of reported speech in argumentative discourse. Languages, 7(59), 1-11. DOI:https://doi.org/10.3390/languages7010059

Hinton, M., & Wagemans, J.H.M. (2022). Evaluating reasoning in natural arguments: A procedural approach. Argumentation, 36, 61-84. DOI:https://doi.org/10.1007/s10503-021-09555-1

Kidd, I. J. (2016). Intellectual Humility, Confidence, and Argumentation. Topoi, 35(2), 395–402. https://doi.org/10.1007/s11245-015-9324-5

Wagemans, J.H.M. (2019). Four basic argument forms. Research in Language, 17(1), 57-69. DOI: https://doi.org/10.2478/rela-2019-0005

Wagemans, J.H.M. (2021). Argument Type Identification Procedure (ATIP) – Version 4. Published online December 30, 2021. URL =www.periodic-table-of-arguments.org/argument-type-identification-procedure 


Back