Three decades of a concerted effort to reduce corruption has not yielded the hoped-for level of change. To move forward with greater success, we need to increase our understanding of what works and what doesn’t work – as well as when, why, and for whom. Evaluations can contribution to filling this knowledge gap. However, only if they are of sufficient quality. Evaluating anti-corruption interventions is not easy. Challenges range from the difficulty of accurately measuring results to the difficulty of proving causation and contribution.
This paper provides theme-specific advice for anyone commissioning or carrying out an evaluation that either exclusively or in part assesses anti-corruption interventions.
Designing an effective evaluation methodology
In addition to making use of general best practice, anyone commissioning or carrying out an evaluation of an anti-corruption intervention should consider the following. First, corruption is both complex and entrenched. It may therefore be appropriate to carry out a complexity-responsive evaluation and to complement established evaluation approaches with emerging methods and tools.
Second, measuring corruption and changes in corruption is not easy. Evaluators often struggle to secure sufficient, accurate, and intervention-specific data. When available, locally generated data, such as administrative statistics and target surveys, is generally best. When gathering primary data, evaluators should prioritise the safety and comfort of participants; carefully choose their words to ensure clarity and avoid unnecessary controversy; use data-gathering methods that allow for open and honest input; tailor the data-collection approach to the type of corruption discussed and the stakeholder questioned; and gather data from a range of sources.
Third, the results of most interventions will be modest in scale. A project cycle is generally too short a time to bring about significant and lasting change, particularly when the conditions don’t favour reform or when corruption is systemic. Evaluators will therefore need to employ a realistic barometer of success – even if this means questioning the ambitiousness of the logframe.
Fourth, it is often difficult to demonstrate the causal link between a specific anti-corruption intervention and concrete outcomes and impacts. Results are often brought about by a multitude of past and present forces. Evaluators may therefore have to settle for demonstrating contribution, rather than attribution.
Corruption is not a disease or deviation. It is the historical baseline. Curbing corruption is therefore a complex, never-ending process. Progress is messy, non-linear, and uncertain. The possibility of unintended consequences and backlashes is ever present. To fully evaluate an intervention, evaluators will need to look beyond the parameters of the project – both in terms of scope and timeline. They will also need to draw on the anti-corruption literature. More concretely, they will need to determine whether there is a problem—strategy match; assess whether the intervention is in line with latest and local thinking on what works; proactively explore unintended consequences and backlashes; and examine the political settlement and level of support for the intervention. Each of these four points are explored in turn.
Anti-corruption interventions should match the problem they are meant to solve. Too often, a simple strategy is employed for a complex problem. Many interventions are also insufficiently tailored. Even when the same corruption problem appears in two different contexts, the same solution may not work in both, as the underlying drivers and wider conditions will vary. Analysing this problem–strategy match affords evaluators the opportunity to explore whether the intervention is likely to achieve the desired results and be sustainable.
The success of an intervention depends in part on whether it is designed and implemented in line with latest and local thinking on corruption and how to curb it. The body of research on what works, when, why, and for whom is continuously growing. Evaluators should draw on this knowledge when assessing an intervention’s relevance, effectiveness, impact, and sustainability.
Powerful individuals and groups often benefit from maintaining the status quo. Evaluations therefore need to explore whether the anti-corruption intervention has led to – or potentially could lead to – unintended consequences and backlashes. Approaches include speaking to third party experts and selecting tools designed to uncover unexpected results.
Like all interventions, an analysis of the sustainability of an anti-corruption intervention entails asking questions such as ‘Does the intervention have sufficient funds to continue?’ However, there is also another level of questions that need to be answered. Key questions include: Did the intervention address the underlying drivers of corruption? Has the gap between the formal and informal rules of the game been closed? Has the intervention (in combination with other reform efforts) succeeded in shifting the landscape to a new corruption equilibrium? And does the political settlement support the intervention and its goals?
If impact is to be rigorously gauged, evaluators need to design a methodology that considers the realities of corruption and anti-corruption measures. It should reflect the facts that corruption and changes in corruption are difficult to measure; progress is not linear; partial success is often all that is feasible; a multi-pronged approach is generally needed; and determining contribution and causality can be difficult. It also requires acknowledging that an intervention can bring about multiple results, as well as different results for different groups; there are multiple pathways to bringing about a reduction in corruption; what works in one situation may not work in another; and the details of a specific anti-corruption tool matter. Many evaluation handbooks favour quantitative methods for evaluating impact. However, qualitative approaches also have their advantages. They are, for example, generally better able to explore unintended consequences. In most cases, a genuinely mixed methods approach is best.
Finally, anti-corruption interventions can help bring about change at a range of levels – from shifting attitudes and building capacity to eliminating a corrupt practice and strengthening deeper systems of accountability in society. The level of results that can be expected will vary based on factors such as the context and the intervention’s scope, scale, and activities. Evaluators need to keep in mind that results are more likely to be sustained if the system dynamic changed (eg the root causes of corruption have been addressed). A focus on transformational changes entails assessing an intervention’s significance, depth of change, and scale of change.
Raising the evaluation bar
Evaluations can contribute to corruption reduction efforts by helping policymakers, managers, and practitioners to design and implement more effective anti-corruption strategies and tools. However, evaluations must be both useful and used. Even with a limited budget and data challenges, it is possible to carry out a high-quality evaluation by recognising and utilising the range of methods and tools available – including those particularly suited for evaluating complex interventions in complex contexts. The methodology, as well as the plan for how the evaluation will be utilised, should promote both learning and accountability.
