Measuring experiences of corruption is essential for several reasons. It allows corruption prevalence to be monitored and understood, helps to meet inter-organisational cooperation requirements – such as tracking progress towards the UN Sustainable Development Goals (SDGs) – and supports decision-making processes. Direct measurements are typically produced through population and business surveys. The robustness, reliability, and representativeness of the measurements depend on the quality of the surveys.

There are several quality issues that survey-based corruption measures might face, such as social desirability and political bias, limited coverage of corruption types, uneven geographical coverage, and the difficulty of comparing across countries and over time, but their evaluation is often discussed in isolation, not comprehensively.

Government representatives and practitioners should use a comprehensive quality assessment framework to test and improve the quality of existing data collection on corruption. It is usually more cost-effective to use and improve existing data collection tools, along with the related data, than to develop new ones.

In the context of corruption measurement, the key questions from a practitioner’s perspective would likely be:

• Is my data collection on corruption relevant for monitoring the SDG Target 16.5?
• Is my data collection accurate and valid in measuring corruption – in general and in my country?

What do we mean by ‘quality’ in evaluating corruption surveys?

According to the ISO guidance on quality management systems (Section 3.6.2), ‘quality’ is ‘the degree to which a set of inherent characteristics of an object fulfils requirements’. The UN National Quality Assurance Framework (UN NQAF) clarifies that, for statistics, quality should be operationalised in a set of dimensions: relevance; accuracy and reliability; timeliness and punctuality; accessibility and clarity; and coherence and comparability.

How can these general quality dimensions be used to assess surveys measuring corruption experiences?

The Quality Evaluation Framework for Sustainable Development Data on Target 16.5 (QUEST_16.5) addresses this need, operationalising ‘quality’ in six dimensions:

1. Relevance: Does the survey provide relevant data for SDG Target 16.5 monitoring?
2. Accuracy: Are corruption concepts accurately operationalised to cover different forms of corruption? Are questions formulated to avoid social desirability bias and ensure follow-ups on corruption events and data disaggregation?
3. Reliability: Does the survey use sound methodologies (in terms of survey mode and sampling strategy) for consistent, stable measures over time and other conditions?
4. Periodicity: Are surveys conducted regularly (at least every three years) for longitudinal trend analysis?
5. Accessibility: Are methodological information, the questionnaire, and the raw data publicly available?
6. Comparability: Does the survey align with international cross-country and cross-time comparison standards?

In this framework, ‘relevance’ refers to the requirement for UN Member States to monitor SDG Target 16.5. This dimension is assessed by checking whether a given corruption survey can collect data on the prevalence of active and passive bribery affecting individuals and businesses. It is still important to note that a survey’s relevance might also be assessed by considering other needs, such as monitoring local corruption issues, that are beyond the scope of the SDGs.

All other dimensions in the framework have been operationalised following more general recommendations for corruption measurement, such as those in the UNODC-UNDP Manual on Corruption Surveys; the UNODC-UNECE Manual on Victimization Surveys; and the UNODC Statistical Framework to Measure Corruption.

In its current form, the framework can be used by national institutions to assess whether their surveys meet SDG requirements to produce internationally relevant data on corruption. It can also be used to understand whether other quality requirements, beyond the SDGs, are met. While primarily focused on measuring the quality of surveys, it can be applied to evaluate the quality of other indicators and data sources.

What are the main quality concerns in corruption surveys?

My colleagues and I evaluated 45 surveys of corruption experiences across 180 countries. These included national and cross-national surveys, specialised and non-specialised surveys, and surveys targeting individuals and businesses. All were conducted between 2005 and 2022. The detailed methodology and results are available in our Sustainable Development paper, but below is a summary of the important findings on each dimension of our framework. A second paper was recently published to evaluate the quality of 13 additional corruption surveys covering 20 countries in the MENA region.

Relevance

Of the 45 corruption surveys, 50% provide data that can be used to monitor SDG Target 16.5. Cross-national surveys outperform national ones, with 78% showing high relevance for SDG monitoring compared to 42% of national surveys. A significant limitation of national surveys is the absence of a screening question to identify respondents who interacted with public officials during a given reference period. This question is necessary to identify those potentially exposed to corruption experiences and thereby to calculate the corruption/bribery incidence. This ratio is crucial for understanding the relative risk of corruption in public service interactions.

Accuracy

Only 31% of surveys exhibit high accuracy in measuring corruption. Indeed, many surveys focus only on passive bribery (ie bribes requested/expected by public officials of citizens). People are more likely to respond honestly to questions around passive bribery, as it avoids the social stigma of active bribery and limits social desirability bias. However, this narrow focus leads to an incomplete, invalid, and skewed picture of the corruption problem, as it limits the coverage of active bribery, nepotism, favouritism, or vote-buying.

Terms like ‘bribe’ or ‘corruption’ are often used to frame questions on the experience of corruption: 23 of the 45 surveys used one or both. These terms are associated with illegal or criminal behaviour, which may discourage people from answering honestly. Furthermore, different respondents might interpret these terms in varying ways, which can reduce the validity of the data and make it harder to compare responses accurately.

Another challenge to accuracy is the limited depth and disaggregation of survey data. Only 47% of surveys include enough follow-up questions about corruption-related events. These questions are essential for breaking down data – for example, by the role or sex of the officials involved – and understanding the incidence of corruption, benefits exchanged, costs of corruption, and whether corruption is reported to the relevant authorities.

Reliability

Of the 45 surveys, 53% show high methodological reliability. The main issues relate to inadequate sample sizes, suboptimal respondent selection, and the lack of a ‘warm-up strategy’ in the questionnaire. This strategy entails starting with simple and non-sensitive questions that can build rapport with respondents before transitioning to sensitive topics like the experience of corruption. Warm-up strategies are considered to be ‘best practice’ for the development of surveys on delicate topics, and without them, respondents may disengage or provide socially desirable answers.

Periodicity

Of the national surveys, 56% have been conducted only once, limiting their utility for trend analysis. By contrast, all the identified cross-national surveys have high periodicity, with repeated waves conducted at least every three years.

Comparability

This is high for 78% of cross-national surveys but for only 39% of national surveys – and is particularly low for non-specialised types. Many national surveys lack standardised methodologies regarding question formulation, respondent selection, and sampling units. While cross-national surveys excel in comparability, their broad focus can flatten local nuances that are better captured by national surveys.

For the full breakdown of our analysis of all 45 selected surveys, and their scores across each dimension, please see the annex. You can also find these in our original published article.

How to improve corruption surveys

1. A key priority is balancing standardisation and contextualisation in corruption survey content. Surveys could adopt a dual-module structure: a core set of standardised questions/indicators to ensure comparability across countries and a set of questions to account for local corruption issues and manifestations. Standardisation can be guaranteed by the use of neutral and operational language referring to exchanges of money, gifts, or favours (as suggested by SDG 16.5) and by avoiding negatively loaded and vague terms. Follow-up questions with colloquial and regionally relevant terminology can further capture cultural nuances. For instance, terms like tangente, corruzione, and concussione in Italy are used to distinguish between passive and active bribery. Similarly, Spanish-speaking surveys use terms like coima, soborno, or mordida, and Arabic-speaking countries refer to wasta to describe specific corruption habits. A mix of sociotropic and individual-oriented questions about specific types of corruption should be considered to support political decisions.
   
2. Surveys should expand their scope to include corruption types beyond passive bribery. For example, Ethiopia has successfully operationalised nepotism and favouritism, while Italy addresses vote-buying and favouritism. The Global Corruption Barometer includes questions on favouritism and the use of personal connections to access public services, as well as on vote-buying and sextortion. The latter can be measured by including ‘sexual favours’ as an option for the range of benefits exchanged during corrupt transactions.
   
3. Conducting recurring survey cycles at least every three years is essential to monitoring corruption trends and evaluating policy impacts. Corruption modules can be integrated into ongoing surveys, such as labour force or household surveys, to improve periodicity without incurring significant costs.
   
4. Encouraging regional surveys and collaborations, like Afrobarometer or AmericasBarometer, can ensure financial support for more regular surveys and cross-country comparability.
   
5. Increasing the availability and accessibility of raw data and methodological details is vital for building trust in survey findings and supporting evidence-based policymaking.
   
6. In this context, maintaining the role and independence of national statistical offices is crucial for ensuring data quality and validation while upholding transparency and accountability. The UNODC Statistical Framework to Measure Corruption highlights the importance of empowering national authorities to lead corruption data collection efforts autonomously. This framework inspires a vision where countries would establish coordination mechanisms to oversee and guarantee quality during the collection, integration, analysis, and dissemination of corruption data at the national level.