A bit of history
As (Innes & Booher, 2000, p.173) claim “Indicators and performance measures have become an important element in policy initiatives relating to sustainability and to the re-invention of government”. The idea of employing quantitative indicators in order to evaluate policy implementation goes back to the ‘40s, when the US economy was being evaluated in terms of the Monthly Economic Indicators (Wong, 2006, p.1ff). The idea of exploiting social indicators and developing a theory for defining, using, combining and interpreting them, passed from the US Administration to the large international organisations such as the United Nations (UN, Social and Economic Council) and the Organisation for Economic Co-Operation and Development (OECD).
The wave of interest around prosperity indicators has been significantly motivated by the global questions on environmental matters and has led to a series of approaches, typically associated to the keywords ‘indicators for quality of life’, ‘sustainability indicators’, sometimes combined with other widely used terms in public discourse, such as `economic competitiveness’, etc. (Sawicki, 2002). As an indication of the widespread interest, let us mention that the European Union has issued a set of recommended ‘European Common Indicators’ focusing on ‘monitoring environmental sustainability at the local level’ while, some years earlier, a call for suitable ‘indicators for sustainability’ had been included in Agenda 21 of the Earth Summit Conference (1992, Rio de Janeiro), which marked an avalanche of actions and initiatives.
One of the major concerns in the construction and exploitation of indicators has been the access to the relevant data and the difficulties in the collection and reliability of the data needed in order to calculate and interpret social metrics. The revolution of the WWW and the Open Data Movement, conceived as “the idea that certain data should be freely available to everyone to use and republish as they wish, without restrictions from copyright, patents or other mechanisms of control” arguably opens a new arena of experimentation with social indicators. This idea lies at the heart of the Policy Compass approach and thus, in this deliverable we will also provide a quick review of the contemporary situation of Open Data.
What is an indicator?
The term ‘indicator’ is one that people can easily understand. It is regularly conceived as a sort of ‘statistical measure’ that can adequately capture crucial aspects of a (social) phenomenon that should be monitored, in particular when a specific policy measure is enforced to affect it. Perhaps then, the simplest and most general definition is that of (Innes J. E., 1990): an indicator is “a set of rules for gathering and organising data so they can be assigned meaning”. In the policy-making arena, an indicator is conceived as a concrete tool used for justifying and optimizing resource allocation. From the scientific perspective, social indicators can be examined both from the theoretical and the practical viewpoint.
A quick look at the typology of indicators
In the preceding subsection, the (abstract) notion of an indicator has been given some well-known definitions. Yet, we should have in mind that it is usually the quantitative nature of indicators which makes them potentially interesting and useful. At this point, it certainly makes sense to see how indicators are perceived by the people who work on their calculation and exploitation, at least technically.
Aggregate (or summary) indicators: An aggregate or summary indicator concentrates information into a single figure. Examples include Gross Domestic Product (GDP) and the Consumers Price Index (CPI).
Composite (or integrated) indicators: Composite or integrated indicators draw from, or reflect, interaction between different areas such as the environmental, economic and social dimensions. An example would be the Human Development Index (HDI). An aggregate indicator can also be a composite indicator. To use this type of indicator successfully, awareness and acceptance of the assumptions that have gone into its construction are required.
Decoupling indicators: Decoupling is a (desired) outcome, such as having reduced energy consumption along with increased economic growth. The decoupling process can be very complex, so indicators aiming to show whether it is happening need to be developed with care.
Headline indicators: Some indicators may be selected as headline indicators – usually because they describe key issues. They are often supported by a subset of indicators. Usually they form a quick guide or overview and can be used to engage public awareness and focus attention. For instance, the UK sustainable development project has 15 headline indicators that are used to make up a quality-of-life barometer. Headline indicators may include composite indicators or other types of indicators, depending on the reporting focus.
International, National, Regional and Local indicators: Indicators are used at all levels, including international, national and regional and may be referred to as national and regional indicators. Indicators can be produced for lower levels such as community scheme monitoring where local indicators may refer to. For example, data gathered at the subnational level to produce regional indicators, could feed into national or international indicator reporting.
Proxy indicators: Proxy indicators are indicators that measure one aspect of a system that is thought to be reflective of a wider system. For example, lichen species are used as a proxy for air quality, and insect species in waterways may be used as a proxy for water quality.
Sustainability and other topic based indicators: Indicators may belong to a set that builds a picture of a whole system or framework, such as sustainability indicators. Sustainable development integrates development and developmental reporting across the economic, environment, cultural and social domains. Sustainability indicators refer to the monitoring of sustainable development.
On the methodology of defining Social Indicators
The description below draws directly from (Wong, 2006, Chapter 7), a very readable presentation. The steps of the methodology comprise:
- Step 1: Conceptual consolidation – Clarifying the basic concept to be represented by the analysis
- Step 2: Analytical structuring – Providing an analytical framework within which indicators will be collated and analysed
- Step 3: Identification of indicators – Translation of key factors identified in Step 2 into specific measurable indicators
- Step 4: Synthesis of indicator values – Synthesizing the identified indicators into composite index/indices or into analytical summary
What makes a `good’ indicator?
According to OECD, a well-defined and useful indicator should comprise (UNEP, 2014):
- Policy relevance: the indicator needs to address issues that are of (actual or potential) public concern relevant to policymaking. In fact, the ultimate test of any single indicator’s relevance is whether it contributes to the policy process.
- Analytical soundness: ensuring that the indicator is based on the best available science is a key feature to ensure that the indicator can be trusted.
- Measurability: the need to reflect reality on a timely and accurate basis, and be measurable at a reasonable cost, balancing the long-term nature of some environmental, economic and social effects and the cyclicality of others. Definitions and data need to allow meaningful comparison both across time and countries or regions.
Innes, J. E. (1990). Knowledge and Public Policy: The Search of Meaningful Indicators,. New Brunswick: NJ: Transaction Publishers.
Innes, J. E., & Booher, D. E. (2000). Indicators for Sustainable Communities: A Strategy Building on Compexity Theory and Distributed Intelligence. Planning Theory and Practice, 1(2), 173-186.
Sawicki, D. S. (2002). Improving community indicator systems: injecting more social science into the folk movement. Planning Theory & Practice, 3(1), 13-32.
UNEP. (2014). GREEN ECONOMY: Using indicators for green economy policy making. Retrieved from http://www.unep.org/greeneconomy/Portals/88/documents/PAGE/IndicatorsWorkingPaper.pdf
Wong, C. (2006). Indicators for Urban and Regional Planning: the interplay of policy and methods. London and New York: Routledge, Taylor and Francis.