30 Sep

Policy Compass at Expertise, evidence and argumentation in the context of public policy-making workshop!

Evidence-based policy-making rests on three pillars that correspond to interlocking phases of the policy-making process. Addressing the relation between evidence, explanation/prediction, and argumentation, the workshop sheded light on how evidence is selected, assessed and presented to support a policy, thus combining communicative, scientific, and political considerations.

Policy Compass partner Fraunhofer FOKUS, represented by Tom Gordon, gave a presentation on the new version of Carneades model of structured argumentation (Version 4), developed in the context of Policy Compass, with better support for cumulative arguments, practical reasoning and multi-criteria decision analysis, applying the insights above, together with web-based software tools using this model to support argument construction, evaluation and visualisation tasks.

30 Jul

Blog Post #03 – Argumentation Support Systems

Argumentation support systems are computer software for helping people to participate in various kinds of goal-directed dialogues in which arguments are exchanged. In the Policy Compass project, our focus is on applying an argumentation support system to help users to argue well about the pros and cons of proposed performance metrics and the reasons or causes of particular performances when applying performance metrics to evaluate performances in the monitoring phase of the policy life cycle.

Argumentation support systems are designed to complement and be used with discussion forums, email, blogs and other computer-based messaging or publishing systems. While messaging and publishing systems provide ways to write and send messages or articles to one or more users, they do not provide specialized tools designed explicitly to support argumentation tasks. For example, they provide no way for a citizen to obtain a quick overview of the issues which have been raised, to list ideas which may have been proposed for resolving such issues, to see in one place the arguments pro and con these proposals, or to get an idea about which positions currently have the best support given the arguments put forward thus far in the dialogue. These are just a few of the kinds of services offered by argumentation support systems.

We have evaluated and compared a number of argumentation support systems currently available, including:

Properties of Argumentation Support Systems

We have evaluated these argumentation support systems along several dimensions:

Dialogue Types.

The main types of dialogue supported: persuasion, information-seeking, negotiation, deliberation, or eristic.

Argument Structure.

The models of argument structure supported: Toulmin (Toulmin 1958), Issue-Based Information Systems (IBIS) (Kunz and Rittel 1970), Beardsley/Freeman, the de facto standard in philosophy (Beardsley 1950; Freeman 1991), abstract, if attack or support relations between arguments are modelled, without modeling the internal structure of individual arguments, as for example in Dung abstract argumentation frameworks (Dung 1995), or graph, if only generic directed graphs are supported with no explicit data model for argument structure. Toulmin and IBIS models can be simulated using Beardsley/Freeman models, but not vice versa. By “structure” here, we mean the underlying conceptual or mathematical model of argument structure, not the diagramming methods for visualizing the structure in argument maps. The same structure can be visualized in various ways.

The forms of metadata supported for annotating elements of the model: plain text, URL, hypertext using a markup language such as HTML or Markdown containing links to source documents, or a structured set of attributes and values, using for example the Dublin Core metadata element set.

Argument Evaluation and Analysis.

The methods of argument evaluation supported: manual, automatic, using a theorem prover or computational model of argument such as ASPIC+ (Prakken 2010), or none.

Argument Construction and Reconstruction

The methods supported for constructing or reconstructing arguments: knowledge-based, using rule-based systems to generate arguments from domain models, schemes, if forms for instantiating argumentation schemes are provided, or manual, if schemes are not used.

Argumentation Schemes

The argumentation schemes supported by the system: none, strict, defeasible, or configurable, if the system provides a language for specifying custom argumentation schemes. Strict schemes are deductively valid inference rules. Defeasible schemes express rules of thumbs, where the premises provides reasons to accept the conclusion, but the conclusion can be defeated by counterarguments.

Argument Visualization and Browsing

The methods supported for visualizing and browsing arguments: argument maps, i.e. two or more dimensional diagrams, hypertext, or none.

Argument Map Layout

The methods supported for laying out argument maps: automatic or manual.

Collaboration Support

Whether the system is a single-user or a multi-user system.

Procedural Support

The methods provided for supporting the procedural aspects of argumentation dialogues: configurable argumentation protocols, commitment stores, or none.

Export Formats

The data formats supported for exporting arguments for use with other systems: Argument Interchange Format (AIF), the Rationale format (RTNL), the Legal Knowledge Interchange Format (LKIF), the Carneades Argument Format (CAF), the GraphML interchange format for directed graphs, Portable Network Graphics (PNG), Portable Document Format (PDF), Structured Vector Graphics (SVG), or none.

Import Formats

The data formats supported for importing arguments from other systems: the Argument Interchange Format (AIF), the Legal Knowledge Interchange Format (LKIF), the Carneades Argument Format (CAF), the GraphML interchange format for directed graphs, or none.

System Type

The type of the system: a desktop application with a graphical user interface, an interactive web application or rich internet application, a set of command line tools, or a service or library, providing an Application Programmers Interface (API) for other applications.

Programming Language

The programming languages used to implement the system

Operating System or Platform

The operating systems (e.g. Linux, Windows, Mac OSX) or platforms (e.g. Java Virtual Machine, JVM) for which the system is available.

Languages

Languages available for the user interface of the system: English, German, Greek, Russian, Spanish, or configurable, if other languages can be supported. Only languages of the partners in the Policy Compass project are listed explicitly here.

Licenses

The software licenses offered by the copyright owners of the system: one or more of the open source licenses certified by the Open Source Initiative, closed-source, if only the object code of the system is available, or none, if neiher the source code nor the object code of the system is available, for example when it is offered only as a service on the Web.

Comparing the Systems

The table below summarizes the properties of the selected argumentation support systems along these dimensions.

ArgumentationSystemsTable

Merely listing properties of system along several dimensions is of coure no substitute for practical experience actually using the systems, to learn more about their maturity, stability, usability, performance and other qualities.

Nonethless, this initial analysis can help us to focus our energies on evaluating more closely systems which appear to be promising, given the requirements of the Policy Compass project. If we limit our choices to open source, multi-user web applications, only AGORA-net, Carneades, Cohere, and LASAD remain to be considered. Of these Cohere and LASAD suffer from a lack of any kind of support for argumentation schemes or argument evaluation and analysis. AGORA-net seems to be easy to use, with a very nice graphical user interface, but supports only deductive (strict) argument schemes and also provides no analysis or evaluation tools.

Carneades appears to provide the most features, lacking only support for the procedural aspects of argumentation dialogues, such as support for argumentation protocols. The ARG-tech suite provides nearly as many features, and is the only system currently which uses protocols to support argument dialogues. But the ARG-tech suite is currently a closed-source system provided only as an online service by the University of Dundee. It is also not clear how well the various tools of the ARG-tech suite are integrated. The user interface of the each tool in the suite has a different look and feel. To be able to evaluate the quality of the implementation of the ARG-tech tools it would be necessary to gain access to the source code.

Carneades of course has the advantage of having been developed by the partner responsible for argument maps in the Policy Compass project, Fraunhofer FOKUS. Thus the code is well understood and can be most easily customized and extended to meet particular Policy Compass requirements.

References

Beardsley, Monroe C. 1950. Practical Logic. New York: Prentice Hall.

Dung, Phan Minh. 1995. “On the Acceptability of Arguments and Its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and N-Person Games.” Artificial Intelligence 77 (2). Essex, UK: Elsevier Science Publishers Ltd.: 321–57.

Freeman, James B. 1991. Dialectics and the Macrostructure of Arguments: A Theory of Argument Structure. Berlin / New York: Walter de Gruyter.

Kunz, Werner, and Horst W.J. Rittel. 1970. “Issues as Elements of Information Systems.” Institut für Grundlagen der Planung, Universität Stuttgart.

Prakken, Henry. 2010. “An Abstract Framework for Argumentation with Structured Arguments.” Argument & Computation 1: 93–124.

Toulmin, Stephen E. 1958. The Uses of Argument. Cambridge, UK: Cambridge University Press.