Development practitioners routinely encounter corruption as a key obstacle to achieving their programming objectives. They confront questions such as: How serious is corruption in the country and sector in which we are working? Is it getting worse or better? What corrupt practices are taking place and where? Why? Who is involved in corruption? Who does corruption harm? Corruption research has responded with a plentiful supply of indices, scores, ranking and assessments to help answer them. However, it is not always clear what these tools really tell us, or how we can or should use them. This Guide explains how corruption and measurements can help solve real-world challenges of designing, implementing, monitoring and evaluating programmes.
- Static analyses, such as integrity system studies and corruption ‘measures’ may identify problems and areas of risk. Dynamic analyses, such as political economy assessments, identify drivers of corruption, as well as opportunities and constraints for addressing them.
- Corruption and reform in a particular sector or function of government may be influenced by factors outside that sector or function. Therefore it is important to explore elements of the broader system (for example public financial management) that can help explain corruption problems.
- Causality and attribution problems make the overall level of corruption an inappropriate outcome or even impact-level indicator.
- Multiple sources of information are usually needed to create a robust evidence base for the evaluation findings.
- Internationally-generated data sources seldom tell us what we need to know for programming. ‘Homegrown’ data such as administrative statistics, targeted surveys and bespoke proxy indicators are almost always more likely to reflect the actual effects of a given programme.
I. Introduction: Using corruption measurement and assessment tools in development programming
Development practitioners routinely encounter corruption as a key obstacle to achieving their programming objectives and confront questions such as: How serious is corruption in the country and sector in which we are working? Is it getting worse or better? What corrupt actions are taking place and where? Who are corruption’s victims, perpetrators, and opponents, and how do the systems in which they operate facilitate, drive, or discourage corruption?
Corruption research has responded with a plentiful supply of indices, scores, rankings, and assessments to help answer them. Assigning scores and creating profiles of legal frameworks, institutional arrangements, and other characteristics that facilitate or constrain corruption is now a major research area, but how should development practitioners use this information to solve the real-world challenges of designing, implementing, monitoring, and evaluating programmes?
This guide is for development practitioners interested in using corruption-related measurements and methodologies to analyse the problem as well as to design and monitor programmatic responses. Section II provides an overview of different types of tools, while Section III covers characteristics of these tools that affect their utility for diagnosis of the corruption problem and designing and monitoring programmatic responses. Section IV provides specific guidance on how development practitioners can use these tools in the daily work of development programming, while Section V summarises some essential lessons. This guide is not an inventory or evaluation of corruption measurement and assessment tools; very good ones already exist.
II. Types of corruption measurement and analytical tools
The analytical tools that programmers will be interested in generally fall into two categories: those that attempt to ‘measure’ corruption, and those that assess contextual factors that contribute to corruption.
- Corruption ‘measurements’ generally attempt to quantify the extent of corruption in different ways.
- Assessments and assessment methodologies, by contrast, aim to describe the characteristics of a given context – system shortcomings (and strengths) and political, economic and social factors – that enable and sustain corruption.
This section summarises the types of measurements and assessment tools available, provides examples, and offers a few considerations on using these tools. Annex 1 includes more information of each example listed in this section, along with many others, including descriptions of their focus, coverage, methodology, strengths, and limitations.
A. ‘Measuring’ corruption
Corruption is often described as a ‘hidden phenomenon,’ though in fact its presence is often common knowledge. Corrupt behaviour is not something that participants willingly expose for just anyone to see – let alone quantify. That is one of the main reasons we are never in a position to accurately ‘measure’ corruption itself, but there are other challenges, too.
What aspects of corruption should one ‘measure’? Different types of corrupt practices – for instance, administrative corruption versus grand corruption – require different approaches due to their differing impact on society. In the case of informal payments for services or sexual exploitation, the scope of incidence is more relevant than the monetary value when considering the immediate impact on women and the poor. With grand corruption, the scope of incidence is perhaps less relevant than the financial amount of bribes and the scale of illicit enrichment of corrupt officials, not to mention the longer-term effect on the economy.
Different corruption ‘measurement’ tools consequently attempt to quantify different types or aspects of corrupt practices. Some count reported victimisation (‘experience’), others survey opinions of experts and broader populations (‘perceptions’), while others track certain types of administrative data (eg, the number of procurements conducted in accordance with key procedures). These different measurements may be grouped generally into countable statistics and indices typically constructed from multiple data sources.
The seduction of quantification
An important debate is emerging about the pros and cons of counting things and creating quantitative measurements to describe complex social phenomena (see Merry 2016). In its pursuit of evidence to demonstrate effectiveness and impact, the international development field has embraced the quantification trend, and the sheer number of measurements in Annex 1 is proof that the anti-corruption field is no exception.
Criticisms of quantification include the fact that such measures usually only partially describe the range of factors that contribute to a certain societal characteristic (eg, respect for human rights, effectiveness of governance, or levels of corruption), that they tend to accentuate things that are ‘countable’ rather than things that are not, and that they may embody biased (or at least hidden) assumptions about how change happens and what is most important for a society.
These criticisms are certainly relevant to corruption measurements, and this guide seeks to highlight the weaknesses of various approaches, along with what they can do. The most important lesson, restated at the end of this guide, is that practitioners must not use corruption measurements uncritically, but rather must explore methodologies and assumptions for themselves and complement measurement with other types of assessment, in order to avoid the worst risks of our attraction to quantification.
Corruption-related statistics: The following types of data are all ‘countable,’ but their resemblance to hard data does not mean they are necessarily good measurements of corruption. (These measurements are not included in the Annex 1 table because they are specific to a given country and not available in any specific format or central information source.)