30 Jul

Blog Post #03 – Argumentation Support Systems

Argumentation support systems are computer software for helping people to participate in various kinds of goal-directed dialogues in which arguments are exchanged. In the Policy Compass project, our focus is on applying an argumentation support system to help users to argue well about the pros and cons of proposed performance metrics and the reasons or causes of particular performances when applying performance metrics to evaluate performances in the monitoring phase of the policy life cycle.

Argumentation support systems are designed to complement and be used with discussion forums, email, blogs and other computer-based messaging or publishing systems. While messaging and publishing systems provide ways to write and send messages or articles to one or more users, they do not provide specialized tools designed explicitly to support argumentation tasks. For example, they provide no way for a citizen to obtain a quick overview of the issues which have been raised, to list ideas which may have been proposed for resolving such issues, to see in one place the arguments pro and con these proposals, or to get an idea about which positions currently have the best support given the arguments put forward thus far in the dialogue. These are just a few of the kinds of services offered by argumentation support systems.

We have evaluated and compared a number of argumentation support systems currently available, including:

Properties of Argumentation Support Systems

We have evaluated these argumentation support systems along several dimensions:

Dialogue Types.

The main types of dialogue supported: persuasion, information-seeking, negotiation, deliberation, or eristic.

Argument Structure.

The models of argument structure supported: Toulmin (Toulmin 1958), Issue-Based Information Systems (IBIS) (Kunz and Rittel 1970), Beardsley/Freeman, the de facto standard in philosophy (Beardsley 1950; Freeman 1991), abstract, if attack or support relations between arguments are modelled, without modeling the internal structure of individual arguments, as for example in Dung abstract argumentation frameworks (Dung 1995), or graph, if only generic directed graphs are supported with no explicit data model for argument structure. Toulmin and IBIS models can be simulated using Beardsley/Freeman models, but not vice versa. By “structure” here, we mean the underlying conceptual or mathematical model of argument structure, not the diagramming methods for visualizing the structure in argument maps. The same structure can be visualized in various ways.

The forms of metadata supported for annotating elements of the model: plain text, URL, hypertext using a markup language such as HTML or Markdown containing links to source documents, or a structured set of attributes and values, using for example the Dublin Core metadata element set.

Argument Evaluation and Analysis.

The methods of argument evaluation supported: manual, automatic, using a theorem prover or computational model of argument such as ASPIC+ (Prakken 2010), or none.

Argument Construction and Reconstruction

The methods supported for constructing or reconstructing arguments: knowledge-based, using rule-based systems to generate arguments from domain models, schemes, if forms for instantiating argumentation schemes are provided, or manual, if schemes are not used.

Argumentation Schemes

The argumentation schemes supported by the system: none, strict, defeasible, or configurable, if the system provides a language for specifying custom argumentation schemes. Strict schemes are deductively valid inference rules. Defeasible schemes express rules of thumbs, where the premises provides reasons to accept the conclusion, but the conclusion can be defeated by counterarguments.

Argument Visualization and Browsing

The methods supported for visualizing and browsing arguments: argument maps, i.e. two or more dimensional diagrams, hypertext, or none.

Argument Map Layout

The methods supported for laying out argument maps: automatic or manual.

Collaboration Support

Whether the system is a single-user or a multi-user system.

Procedural Support

The methods provided for supporting the procedural aspects of argumentation dialogues: configurable argumentation protocols, commitment stores, or none.

Export Formats

The data formats supported for exporting arguments for use with other systems: Argument Interchange Format (AIF), the Rationale format (RTNL), the Legal Knowledge Interchange Format (LKIF), the Carneades Argument Format (CAF), the GraphML interchange format for directed graphs, Portable Network Graphics (PNG), Portable Document Format (PDF), Structured Vector Graphics (SVG), or none.

Import Formats

The data formats supported for importing arguments from other systems: the Argument Interchange Format (AIF), the Legal Knowledge Interchange Format (LKIF), the Carneades Argument Format (CAF), the GraphML interchange format for directed graphs, or none.

System Type

The type of the system: a desktop application with a graphical user interface, an interactive web application or rich internet application, a set of command line tools, or a service or library, providing an Application Programmers Interface (API) for other applications.

Programming Language

The programming languages used to implement the system

Operating System or Platform

The operating systems (e.g. Linux, Windows, Mac OSX) or platforms (e.g. Java Virtual Machine, JVM) for which the system is available.

Languages

Languages available for the user interface of the system: English, German, Greek, Russian, Spanish, or configurable, if other languages can be supported. Only languages of the partners in the Policy Compass project are listed explicitly here.

Licenses

The software licenses offered by the copyright owners of the system: one or more of the open source licenses certified by the Open Source Initiative, closed-source, if only the object code of the system is available, or none, if neiher the source code nor the object code of the system is available, for example when it is offered only as a service on the Web.

Comparing the Systems

The table below summarizes the properties of the selected argumentation support systems along these dimensions.

ArgumentationSystemsTable

Merely listing properties of system along several dimensions is of coure no substitute for practical experience actually using the systems, to learn more about their maturity, stability, usability, performance and other qualities.

Nonethless, this initial analysis can help us to focus our energies on evaluating more closely systems which appear to be promising, given the requirements of the Policy Compass project. If we limit our choices to open source, multi-user web applications, only AGORA-net, Carneades, Cohere, and LASAD remain to be considered. Of these Cohere and LASAD suffer from a lack of any kind of support for argumentation schemes or argument evaluation and analysis. AGORA-net seems to be easy to use, with a very nice graphical user interface, but supports only deductive (strict) argument schemes and also provides no analysis or evaluation tools.

Carneades appears to provide the most features, lacking only support for the procedural aspects of argumentation dialogues, such as support for argumentation protocols. The ARG-tech suite provides nearly as many features, and is the only system currently which uses protocols to support argument dialogues. But the ARG-tech suite is currently a closed-source system provided only as an online service by the University of Dundee. It is also not clear how well the various tools of the ARG-tech suite are integrated. The user interface of the each tool in the suite has a different look and feel. To be able to evaluate the quality of the implementation of the ARG-tech tools it would be necessary to gain access to the source code.

Carneades of course has the advantage of having been developed by the partner responsible for argument maps in the Policy Compass project, Fraunhofer FOKUS. Thus the code is well understood and can be most easily customized and extended to meet particular Policy Compass requirements.

References

Beardsley, Monroe C. 1950. Practical Logic. New York: Prentice Hall.

Dung, Phan Minh. 1995. “On the Acceptability of Arguments and Its Fundamental Role in Nonmonotonic Reasoning, Logic Programming and N-Person Games.” Artificial Intelligence 77 (2). Essex, UK: Elsevier Science Publishers Ltd.: 321–57.

Freeman, James B. 1991. Dialectics and the Macrostructure of Arguments: A Theory of Argument Structure. Berlin / New York: Walter de Gruyter.

Kunz, Werner, and Horst W.J. Rittel. 1970. “Issues as Elements of Information Systems.” Institut für Grundlagen der Planung, Universität Stuttgart.

Prakken, Henry. 2010. “An Abstract Framework for Argumentation with Structured Arguments.” Argument & Computation 1: 93–124.

Toulmin, Stephen E. 1958. The Uses of Argument. Cambridge, UK: Cambridge University Press.

15 Jul

The 3rd Issue of our Newsletter is out!

The third issue of the Policy Compass newsletter has just been published!

The newsletter provides a glimpse at the following issues:

  • use of Fuzzy Cognitive Maps (FCMs) in Policy Compass,
  • participation of Policy Compass in various events,
  • upcoming events of interest etc.

Feel free to download it here (.pdf) and disseminate it to any stakeholders you consider relevant!

10 Jul

Blog Post #02 – Fuzzy Cognitive Map for Policy Modeling and Simulation

Policy making is a group process in which multiple stakeholders identify issues, develop alternative options to resolve the issues, collect citizens’ opinions on the options through public consultation, and publish new changes in policies. Due to the nature of the tasks of the process, policy makers in practice employ unstructured decision making methods including focused group interviews, the Delphi method, and nominal techniques. While such methods have the advantage of collecting qualitative and subjective expert opinion, they have limitations in supporting evidence based policy making which require objective and quantitative data that exists in public on various data sources.

Fuzzy Cognitive Maps (FCMs) allow decision makers to integrate domain knowledge from multiple experts and conduct ‘what-if analysis’ through simulation. Fuzzy cognitive maps (FCMs) have been widely applied to various applications including engineering, industrial marketing etc. FCMs were introduced by Bart Kosko in 1986, where he developed a way to structure expert knowledge using a software programming approach. An FCM can be understood as a graphical representation of the knowledge or the perception of a given model. An FCM is a combination of Fuzzy logic and cognitive map (CM). A CM consists of concept nodes, causal relation and causal relation weighting factor. The causal relation weighting factor is assigned with directed fuzzy value that shows the strength of the causal condition between concepts. This feature marks one of the primary differences between a FCM and a CM.

In the Policy Compass project, FCMs will be applied in two pilot cases; one of a local and one of a regional scope. The lines below focus on the local pilot case of Policy Compass (Cambridgeshire County Council – CCC), while details on the second pilot case (St. Petersburg) will be provided in a future blog post.

In Cambridgeshire County Council, FCMs will be applied to the community learning and skill development (CLSD) funding issue. Cambridgeshire County Council has to respond to the UK Government policy on community learning (which is focused on assisting skills development within the local community) on a regular basis. For this purpose, the government allocates financial resources to the council through a Community Learning Fund which is managed by the national Skills Funding Agency (SFA). The council responds to this public policy by assigning a Community Learning Trust (CLT) Fund which is used to distribute resources to local training agencies that specialise in adult learning. The CLT aims to commission, deliver and support learning in ways that contribute directly to the objectives below, including:

  • bringing together people from all backgrounds, cultures and income groups, including people who can/cannot afford to pay
  • using effective local partnerships to bring together key providers and relevant local agencies and services
  • devolving planning and accountability to neighbourhood/parish level, with local people involved in decisions about the learning offered
  • involving volunteers and Voluntary and Community Sector organisation (VCSO) groups, shifting long term, ‘blocked’ classes into learning clubs, growing self-organised learning groups, and encouraging employers to support informal learning in the workplace
  • supporting the wide use of online information and learning resources
  • minimising overheads, bureaucracy and administration.

To achieve the aforementioned objectives, the CLT objectives are defined in the Cambridgeshire County Council Adult Learning and Skills Strategy (Skills strategy framework). The skills strategy is implemented through different action plans according to local priorities in four different districts in Cambridgeshire. Each district has Community Learning and Skills (CLAS) partnership which identify local priorities for funding. The priorities for each district are identified annually by Partnership members using a range of information such as DBIS policy, SFA funding rules, CCC skills strategy, data on deprivation, unemployment, current availability of provision, historical provision, local knowledge of stakeholders, facilities etc. This process is identified in the action plan as a local needs analysis. The funding decision is made based on scorecards which are marked by proposal evaluators.

The problems with the current process for CLT funding include the lack of ‘learner voice’ in the decision making and local Learner Advisory Panels (LAP) are being developed to address this. Also, the priority setting in local district is still conducted based on qualitative opinion of participants despite of the existence of quantitative data due to the lack of analytic tool. Also, the evaluators of proposals are lacking tools to conduct direct impact analysis of the proposals toward the local priorities and skills strategy.
Policy Compass will provide decision makers with user friendly graphic interface for analysing different indices in comparison with multiple regions within the district. Also, the policy model based on fuzzy cognitive map is expected to allow the proposal evaluators to conduct impact analysis which shows how much impact a proposal can make to the local priorities and skills strategy of the council.

The following figure illustrates how FCM can be applied to model the policy impact of funding proposals. The concepts on the right hand side include strategic and goal concepts while the ones on the left include the concepts derived from submitted proposals. The diagram show how each selected funding proposal can make impacts to the strategic goals of the programme.

FCM

Moreover, the following figure shows a potential simulation result which will show how the strategic goal concepts will be changing during the simulation periods when one of the proposal concept on the left hand side is selected. The graphs will show how strategic goal concepts will be changing by funding of one or more proposals.

FCM2

 

10 Jul

Policy Compass in t-Gov 2014!

The aim of the t-Gov workshop was to provide a common platform for academics and practitioners to discuss and present original research highlighting issues related with technical, organisational, managerial and socioeconomic aspects of both (e) and (t)-Government implementation and adoption.

IMG_4928
Brunel University presented a scientific paper in the context of t-Gov 2014 (entitled “Evidence-Based, Transparent and Accountable Policy Analysis and Evaluation – The Policy Compass Approach”), relevant to the project’s concept and approach.

IMG_4930

The relevant presentation can be found here.

 

10 Jul

Policy Compass in Samos Summit 2014!

Being recognized as one of the established and important summer events of the last few years, this year’s Samos Summit was dedicated on ICT-enabled Governance.
Since the scope of Samos Summit 2014 totally fell in the interests and areas of application of the Policy Compass project, the Policy Compass consortium was represented by Fraunhofer FOKUS in the event. The project’s scope, as well as the up to date findings and advancements were communicated to a large and multidisciplinary audience.

samos summit

In addition, exchange of ideas and knowledge and possible synergies among related projects and initiatives were discussed.

The respective presentation can be found here.

04 Jul

Policy Compass 2nd Plenary Meeting @ ATOS, Spain

The 1st Plenary Meeting of the Policy Compass consortium took place in the ATOS premises in Madrid, Spain on the 26th and 27th of June, 2014.

A sum up of the up-to-date achievements of the project took place, as well as interesting and fruitful discussions regarding the project’s architectural design, evaluation methodology and pilot applications. Discussions on the project’s publications and dissemination activities were also realised.

CAM00237

Stay tuned for more details, as well as for the respective presentations!