Accuracy matters

Corruption and its consequences are inherently difficult to measure. This is due in part to corruption’s clandestine nature. But it also reflects the fact that the term encompasses a wide range of acts, from bribery and embezzlement to sextortion and nepotism. Despite the measurement challenges, many of those contributing to the anti-corruption fight recognise the usefulness of expressing some basic dimensions of corruption in quantitative terms. Quantitative measurement can, for example, make the problem seem more concrete and give a sense of its scale in both absolute and relative terms.

Leading international organisations, donor agencies, and civil society groups frequently foreground global corruption statistics in their public-facing reports, advocacy documents, and speeches. These descriptive statistics are not always, or even usually, essential to the larger claims being advanced by these organisations. Sometimes statistics are cited merely to underscore the magnitude of the corruption problem, with the exact number mattering less than the fact that the number seems big. Indeed, corruption statistics – particularly those that seek to characterise corruption or its consequences at a global level – often function more as a call to action than as a guide to action. When this is so, their source and precision may not matter much to those who invoke them; after all, the harmful impact of corruption has been rigorously documented.

Nonetheless, these quantitative factual claims are worth closer scrutiny for at least two reasons. First, insofar as corruption statistics are being used to inform advocacy and policy decisions or set policy priorities, it is important to ensure that they are reasonably reliable. Second, given the emphasis that the anti-corruption community has placed on evidence-based policy and evidence-based advocacy, the community should set a good example – and ensure its own credibility – by holding the quantitative evidence that it presents to high standards, even with respect to statistics that are not directly used for designing or measuring specific policy interventions.

The goal of this Issue is to provide more transparency and clarity about the origin and reliability of some of the most frequently cited global corruption statistics, together with an evaluation of whether those statistics are sufficiently credible to be presented by leading organisations in their public communications. We have not attempted a comprehensive survey of all factual claims about corruption in the policy and advocacy literature. Rather, we have selected corruption statistics that have appeared prominently in speeches, fact sheets, issue briefs, and similar documents from major organisations. We have also limited our analysis to ten statistics that require some degree of estimation, extrapolation, or generalisation. For each of these ten statistics, we attempted to trace the number back to its origin and to assess the credibility and reliability of the estimate. Based on this assessment, we assigned each of the reviewed statistics one of three grades: credible, problematic, or unfounded.

The results of our investigation are disheartening. Of the ten corruption statistics we assessed, not a single one could be classified as credible, and only two (an estimate of illicit financial outflows and an estimate of corruption’s impact on child mortality) even came close to credibility. Six of the ten statistics are problematic, some seriously so, and the other four are, as far as we can tell, entirely unfounded. Our research also revealed that most of the statistics we investigated were produced one to two decades ago. Poor referencing has, however, given them the appearance of being more recent.

These findings do not mean that the problem these statistics are meant to capture is not real and important. We do not doubt that it is. Furthermore, our critical assessment of these statistics should not be mischaracterised as a finding that these numbers are exaggerations or overstatements. Indeed, it is quite possible that in some cases they significantly understate the harm they are meant to describe. The problem is that we do not know. We believe that the anti-corruption community can and should do better in its treatment and presentation of quantitative evidence, and though much of the discussion in this Issue is critical, it is meant in a constructive spirit.

This Issue is organised as follows. In the next section, we provide an overview of our method for selecting and assessing corruption statistics. We then present our findings with respect to each of the ten statistics investigated. A brief discussion offers some reflections on the possible reasons for the prevalence of problematic and unfounded corruption statistics in the statements and reports of leading organisations. Finally, we suggest five rules of thumb that organisations can use to improve the credibility and validity of the factual claims about corruption that they present to the interested public.

Selection and assessment method

The first step in this project was to select a manageable number of corruption statistics to investigate. We did not aspire to provide a comprehensive assessment of the reliability of all corruption statistics cited in public discussions, nor to make general claims based on a representative sample. Rather, we set out to identify a handful of statistics that seemed sufficiently prominent to be worthy of scrutiny. To do this, we proceeded in two steps. First, using a combination of keyword searches and snowballing, we identified 71 potentially relevant quantitative statistics from a range of sources. From this list, we selected statistics for more in-depth review based on the following criteria:

• The factual claim is about corruption, or a particular form of corruption (such as bribery or embezzlement), or the consequences of corruption.
• The factual claim is expressed in quantitative terms (for example, dollar amounts or percentages).
• The factual claim is made at the global level, rather than about specific countries or regions. We imposed this criterion purely to keep our assessment exercise manageable. Follow-up research can and should assess regional or country-level corruption statistics.
• The factual claim is the product of some attempt at estimation, extrapolation, or generalisation, rather than simply a report of the results of a specific survey or similar measurement exercise. We applied this criterion not because survey results and similar data are unimportant, but because our goal in this Issue is to assess statistical claims that require some degree of inference or analysis.
• The factual claim has been cited within the last decade by at least two major organisations in their public-facing documents. These include multilateral organisations such as the World Bank, Organisation for Economic Co-operation and Development, International Monetary Fund, or United Nations; government donor agencies such as the US Agency for International Development, UK Department for International Development, or Germany’s GIZ; and prominent international non-governmental advocacy groups such as Transparency International or the UN Global Compact.

We applied the last criterion because we are most interested in corruption statistics that feature prominently in the public statements of organisations that command a large global audience (even though, as we detail below, some of the statistics originate in, and are cited by, research papers or similar documents). In some cases the statistic appeared on the organisation’s website without a date; we treated those statistics as having been cited by the organisation within the last decade, given that the statistic still appears on the website. We acknowledge variation in how organisations present these factual claims. Some present quantitative statistics as known facts, while in other cases they include qualifying language – something along the lines of ‘Though the problem is difficult to quantify, some estimates indicate …’. Where such qualifiers were present, we counted the organisation as having cited the statistic if it appeared to be presenting the statistic as sufficiently reliable to inform policy and advocacy decisions.

Using this approach, we narrowed our original list of 71 statistics to the following ten, which are the focus of our analysis:

1. Approximately US$1 trillionb60db59ffff5 in bribes is paid worldwide every year.
2. Approximately US$2.6 trillion in public funds is stolen/embezzled every year.
3. Corruption costs the global economy approximately US$2.6 trillion, or 5% of global GDP, each year.
4. Corruption, together with tax evasion and illicit financial flows, costs developing countries approximately US$1.26 trillion each year.
5. Approximately 10%–25% of government procurement spending is lost to corruption each year.
6. Approximately 10%–30% of the value of publicly funded infrastructure is lost to corruption each year.
7. Approximately 20%–40% of spending in the water sector is lost to corruption each year.
8. Up to 30% of development aid is lost to fraud and corruption each year.
9. Customs-related corruption costs World Customs Organization members at least US$2 billion per year.
10. Approximately 1.6% of annual deaths of children under 5 years of age (over 140,000 deaths per year) are due in part to corruption.

We attempted to trace each of these statistics back to its original source. Sometimes this was straightforward, but often it was not. When we were uncertain as to the origin of the statistic – for example, because it was cited in a speech that did not provide sources – we expanded our research to see whether we could make an educated guess, based on seemingly related statistics mentioned in other documents, as to the original source.

We recognise that some may question the propriety of such guesswork, but we thought it was appropriate for two reasons. First, it would be unfair and misleading to condemn certain corruption statistics as unfounded if we are able to find a credible source that supports the claim being evaluated. Second, informed speculation about how certain quantitative claims migrated from their original sources to the public claims of prominent organisations might stimulate useful discussions about the origins and use of anti-corruption statistics. In the body of the paper, we strive to be transparent about when we are making educated guesses, and about the reasons for those guesses; readers who are uncomfortable with this aspect of our assessment may discount or ignore these passages.

We then assessed the credibility and reliability of the estimate in question and assigned each statistic one of three grades, according to the criteria outlined below.

Credible

We deemed a corruption statistic ‘credible’ if we could trace the statistic to an analysis that employed a transparent and plausible estimation method, as well as reasonably reliable data. We did not require scientific certainty (an unattainable standard in any event) to classify a corruption statistic as credible. After all, given our focus on statistics that are based on estimation or extrapolation, a degree of uncertainty is inevitable.

Furthermore, while a substantive assessment of the data and methods used to generate the statistics in question was an inherent feature of our assessments, we recognise that the existence of serious substantive critiques of the methodology or data should not disqualify a statistic from earning a ‘credible’ classification. After all, virtually all statistics that are based on some form of extrapolation or estimation are vulnerable to methodological challenge. We therefore had to make some admittedly subjective judgement calls as to when concerns about the data or methods are sufficient to render a statistic non-credible. In the discussion below, we try to be as transparent as possible about our reasoning so that readers may form their own conclusions about whether our designations are appropriate. A credible statistic is one that, in our view, is based on sufficiently solid data analysis that organisations should feel comfortable citing the statistic (with proper attribution) in their public-facing documents.

Problematic

We deemed a corruption statistic ‘problematic’ if, after tracing the statistic to its likely origin, we identified serious problems with the data and/or analytical techniques used to generate the statistic in question. For example, we would label a corruption statistic as problematic if it is based on unexplained or unjustified extrapolations from a specific and likely non-representative sample; or if generating the statistic required making very strong, and facially implausible, assumptions about the data; or if the origin of the statistic appears to consist mainly of subjective perceptions by observers lacking sufficient first-hand knowledge. We would also label a corruption statistic as problematic if that statistic, though presented in the form of a single number, is actually the midpoint or high or low end of a very large range in the original estimate.

Organisations should rarely if ever cite problematic corruption statistics without appropriate qualifications. The original data and analysis might still be useful and worth citing, but organisations should take care to cite the empirical findings accurately.

Unfounded

We deemed a corruption statistic to be ‘unfounded’ if we were unable to locate an original source for the statistic, or if the original source appeared to be an unsubstantiated guess, or if the statistic in question is based on a blatant and obvious mischaracterisation of the original source. The conclusion that a corruption statistic is unfounded does not mean that it is necessarily wrong. It does mean, however, that the statistic lacks a credible foundation. Reputable organisations should never cite unfounded statistics. They should instead seek out other qualitative and quantitative information that will allow them to convey their message effectively.

In cases where we tentatively concluded that a statistic was unfounded, or where guesswork was required on our part, we attempted to ascertain the source of the statistic by contacting, via email, individuals and organisations that we thought might have additional information, especially those responsible for the citation of the statistic in public documents. When we received replies, we modified our original draft as appropriate. While we are grateful to the many people who responded to our email queries, we emphasise our belief – which we restate in our conclusion – that it ought to be possible to trace the source of a statistic cited in a public document by a prominent organisation using the citations in the document itself, or some other easily accessible source, rather than having to rely on private email correspondence with the document’s authors.

An assessment of ten global corruption statistics

Corruption statistic no. 1: Approximately US$1 trillion in bribes is paid worldwide every year.

Who cites it: The ‘US$1 trillion in annual bribe payments’ claim has been repeated frequently by various United Nations (UN) officials, organs, and agencies, including Secretary-General António Guterres, the UN Office on Drugs and Crime (UNODC), and the UN Development Programme (UNDP). The statistic has also been used in speeches and publications from the Organisation for Economic Co-operation and Development (OECD) and the UN Global Compact. U4 publications have also cited this statistic.

Origin of this statistic: Most of the organisations that cite ‘US$1 trillion in annual bribes’ do not list a source, or they list the source as the World Bank or World Bank Institute without further detail (that is, without citing a specific document or dataset). The origin of the statistic nevertheless appears relatively clear: it derives from an analysis performed in the mid-2000s by Daniel Kaufmann, then director of global programmes at the World Bank Institute. The methodology, described by Kaufmann in a 2005 paper (subsequently published in the World Economic Forum’s Global Competitiveness Report 2005–2006), is summarised below.

Kaufmann and his research team relied on data from two surveys of firm managers: the 2000 World Bank Enterprise Survey (WBES), which covered 81 countries, and the World Economic Forum’s 2004 Executive Opinion Survey, which covered 104 countries and provided results for the 2004 Global Competitiveness Report. They also drew on 16 separate household surveys conducted by the World Bank between 1999 and 2003. The firm-level surveys asked managers to estimate (a) the amount that firms like theirs pay each year in bribes to government officials, expressed as a percentage of the firm’s total annual sales, and (b) for firms that contract with government, the percentage of the contract value that firms like theirs typically pay as kickbacks to secure public contracts. The household surveys asked respondents to estimate the share of their household income paid in bribes each year.

This survey data, though useful, is insufficient to generate a global estimate of annual bribe payments. To produce such an estimate, Kaufmann and his team extrapolated from the survey results. The extrapolation method is not entirely transparent, and to the best of our knowledge, a full description of the method and the specific calculations has never been made public. The 2005 paper seems to be the fullest public explication of the methodology, and so we rely on that summary here.c74b56d2161c

To extrapolate from household surveys in 16 countries to the entire world, Kaufmann’s team regressed the household survey results against those countries’ scores on the Control of Corruption Indicator (CCI), part of the World Bank’s Worldwide Governance Indicators. The CCI is a composite index that estimates corruption control levels from perception data and other surveys. The regression coefficients were then used to estimate household bribe shares in all other countries. Critically, the analysis assumes that two countries with the same CCI score will have (on average) the same percentage of household income paid in bribes. The analysis further assumes that there is a linear correlation between the CCI score and annual household bribe payments, such that one can reasonably estimate a country’s likely average household bribe payments from its CCI score, even if there are no household surveys from countries with a similar CCI score.

These ‘household bribe share’ estimates were generated for every country for which a CCI score is available (almost every country in the world). Each country estimate was then multiplied by 70% of the country’s gross domestic product (GDP), on the assumption that the ratio of personal consumption to GDP is approximately 0.7. The source of this ratio is not presented, nor is there any explanation for the assumption that the ratio is the same in all countries. In fact, according to OECD data, although the household consumption/GDP ratio in the United States is approximately 0.7, there is considerable variation in this ratio, and the United States is on the high end.

The process for calculating bribe payments by firms was similar. For what Kaufmann calls ‘administrative bribery’ (meaning, so far as we could tell, bribery by firms outside the procurement context), the WBES data on firms’ estimates of informal payments to public officials was converted to an absolute amount in monetary terms. This appears to have been done by multiplying estimates of bribes as a percentage of annual sales by the respondent’s estimate of their firm’s annual sales, weighted by the country’s per capita GDP. This result was then multiplied by the country’s GDP, net of the amount of GDP attributable to government procurement. That product was then multiplied by 0.7 (based on the unsourced assertion that the assumed contribution of business to overall GDP is 0.7). It appears that bribery in public procurement was estimated similarly, although this is not entirely clear from the summary.

As noted above, the precise methods used for estimating bribery amounts are not fully explained. Kaufmann emphasised that he and his team explored ‘138 scenarios’, with different assumptions and extrapolation techniques, including ‘48 scenarios based on the WBES, and 90 scenarios based on the [household surveys]’. They summarised their results as follows: ‘From the 138 scenarios used, if one were to leave out the extreme ‘tails’ (5 percent in each tail), the range of (reasonable) estimates would [be] from US$604 billion to US$1.76 trillion . . . with a reasonable midpoint being close to US$1 trillion’ (emphasis in original).

Our assessment: PROBLEMATIC.

Despite the frequency with which the ‘US$1 trillion in annual bribes’ statistic is cited, we do not think it is sufficiently well grounded to merit repetition in the reports and public declarations of reputable organisations.

First, as emphasised above, although Kaufmann’s 2005 paper provides a general description of the calculation methods, neither the actual data nor a full description of the methodology has yet (to our knowledge) been made publicly available. A more detailed paper, listed as forthcoming in the bibliography to the Kaufmann 2005 piece, was apparently never released, even as a working paper. While we do not question the care and good faith of the research team that produced these estimates, failure to make the methodology fully transparent is not consistent with best practices. That reason alone suggests the need to treat the results with scepticism.

Second, the description of the methodology summarised above raises serious concerns. The assumption that average estimated household bribe payments correlate linearly with CCI scores is dubious: the CCI incorporates multiple forms of corruption, and the relationship between ‘petty’ household bribe payments and overall perceived corruption at the national level may well be non-linear. Furthermore, the assumptions that household consumption equals 70% of GDP and that business contributes 70% of GDP are unsubstantiated and likely incorrect for most countries. Even the assumptions that firms accurately estimate bribes as a percentage of annual sales and that households accurately estimate their own bribe payments as a percentage of their household income may not be correct. More generally, the assumptions required to convert responses on firm and household bribery surveys into global estimates of aggregate bribe transactions are heroic at best, and wildly implausible at worst.

Third, the estimates are severely outdated, relying as they do on survey data from between 1999 and 2005, with most of the data from closer to 2000. Even if the global estimate were accurate back in 2005, at this point it is at least 15 years out of date. If nothing else has changed, that would imply that the US$1 trillion figure is probably a huge underestimate, given that global GDP has more than doubled since 2000 – from approximately US$33.6 trillion to approximately US$87.8 trillion. Of course, other things may have changed as well, including the results of firm and household surveys. There is probably little value, at this point, in citing a figure that is almost two decades old.

Fourth, although the Kaufmann 2005 paper is careful to emphasise that the researchers explored a variety of scenarios and produced a range of bribery estimates between US$604 billion and US$1.76 trillion, much of the subsequent citation of the statistic (including by Kaufmann himself) has presented the estimate as an unqualified ‘US$1 trillion in annual bribes’, without acknowledging the extreme uncertainty associated with that figure. The Kaufmann 2005 paper notes that US$1 trillion is approximately the midpoint of the estimated range, but there is no explanation of why, in this case, the midpoint would be the most likely estimate, nor why the assumptions that generate that particular estimate are more plausible than other assumptions.

For these reasons, while we admire the efforts of Kaufmann and his team to use the available data to generate a global bribery estimate, and we recognise some value in their initial forays into this area, we conclude that there is not a sufficient evidentiary basis to justify the statement, even in passing, that approximately US$1 trillion in bribes is paid each year. The most one could legitimately say, at this point, is that ‘some estimates in the early 2000s found suggestive evidence that the amount paid in bribes each year was probably somewhere between US$600 billion and US$1.76 trillion’. And even that would be pushing the limits of what one can credibly conclude from the study.

One additional observation: An IMF staff discussion note cites an unpublished attempt, in 2015, to update this statistic using the same methods. This revised estimate puts the annual cost of bribery at ‘about US$1.5 to US$2 trillion (roughly 2 percent of global GDP)’. All of the concerns discussed above, with the possible exception of the out-of-date global GDP figures, would apply with equal force to this re-estimate.

Corruption statistic no. 2: Approximately US$2.6 trillion in public funds is stolen/embezzled each year.

Who cites it: The claim that approximately US$2.6 trillion is stolen or embezzled each year has been cited by senior leaders at the UN and the African Development Bank.

Origin of this statistic: It is hard to figure out where this estimate might come from. Its most prominent public citation seems to have been a 2018 speech by UN Secretary-General António Guterres. That speech did not cite a source, but another UN document lists a 2008 joint statement by a coalition of civil society organisations as the authority for this statistic. The civil society coalition’s joint statement, however, purports to provide an estimate of the overall annual cost of corruption (an estimate we will discuss and assess under corruption statistic no. 3 below), rather than an estimate of the annual amount stolen or embezzled. Our best guess as to what happened – and we acknowledge that this is only a guess – is that the speechwriter who inserted the ‘US$2.6 trillion’ estimate into the secretary-general’s 2018 speech simply misunderstood what this figure was supposed to estimate, or did not understand the difference between the cost of corruption and the amount stolen. Then, after the statistic had appeared in a speech by the UN secretary-general, it was cited as authoritative by other officials and organisations.

Our assessment: UNFOUNDED.

To the best of our knowledge, no organisation or researcher has even purported to estimate the annual amount corruptly stolen or embezzled at US$2.6 trillion. The recent appearance of that statistic in speeches and reports from leading organisations, as well as in some media commentary, appears to be due entirely to a misinterpretation or misrepresentation of a statistic on a related but distinct matter in a 2018 speech by the UN secretary-general. (By extension, we suspect that the related claim that ‘the annual costs of international corruption amount to a staggering US$3.6 trillion in the form of bribes and stolen money’ is likely unfounded as well, as it appears to combine the ‘US$2.6 trillion stolen’ estimate and the ‘US$1 trillion in bribes’ estimate.)

Corruption statistic no. 3: Corruption costs the global economy approximately US$2.6 trillion, or 5% of global GDP, each year.

Who cites it: The claim that the aggregate annual economic cost of corruption is roughly US$2.6 trillion, or 5% of global GDP, has appeared in speeches and reports from the UN and the OECD. U4 publications have also cited this statistic.

Origin of this statistic: The source for this claim is obscure. The earliest prominent mention of the US$2.6 trillion figure appears in a 2008 publication, entitled Clean Business Is Good Business, from a coalition of organisations including the International Chamber of Commerce, Transparency International, the World Economic Forum (WEF), and the UN Global Compact. That document declares, in a brief ‘Facts & Figures’ section, that ‘estimates show that the cost of corruption equals more than 5% of global GDP (US$2.6 trillion), with over US$1 trillion paid in bribes each year’. These and other statistics in the document are attributed generally to the World Bank, but no specific source is cited.

The fact that the same bullet point that includes the ‘US$2.6 trillion annual cost of corruption’ statistic also mentions the ‘US$1 trillion in annual bribes’ statistic suggests that the original source may be the estimates produced by Daniel Kaufmann and his team at the World Bank Institute, discussed earlier. However, the Kaufmann 2005 paper does not include any discussion of an attempt to estimate the cost of corruption. The focus of that paper is entirely on estimating the magnitude of corruption – more specifically, the amount paid in bribes each year, expressed in monetary terms. As noted previously, that (problematic) estimation method produced a range of estimates for annual bribery from US$604 billion to US$1.76 trillion.

The Kaufmann paper goes on to discuss what it describes as ‘external checks and validation’ to assess whether this estimate of annual bribe payments seems plausible. (The paper asserts that if the estimated amount of bribe payments is in the same general vicinity as estimates of other sorts of illicit activity, this should increase confidence in the bribe estimates. That assumption is questionable, but it is not the issue here.) One of those ‘external checks’ referenced two earlier estimates of the amount (not the cost) of worldwide money laundering. The first of those sources, a 1998 speech from the IMF’s then managing director, asserted a probable ‘consensus’ that this amount might be somewhere between 2% and 5% of global GDP (or between US$600 billion and US$1.5 trillion at the time). The second source, a 1999 study, estimated the total amount of annual money laundering at about US$2.8 trillion.

Neither of these estimates has anything to do with estimating the annual costs of corruption. So what happened? We do not know, but we can hazard a guess. It is possible that someone who was drafting the 2008 ‘Clean Business Is Good Business’ document skimmed the Kaufmann 2005 paper, saw the 2%–5% of global GDP estimate, failed to realise that this estimate was for the magnitude of global money laundering rather than for the cost of global corruption, and then took the 5% figure (the high end of the GDP range), multiplied it by 2006 GDP (which was about US$51.5 trillion), and came up with approximately US$2.6 trillion. Again, we have no direct evidence that this is the explanation, but it is our best guess. We have not been able to locate a more reliable source for the US$2.6 trillion cost-of-corruption estimate.

Our assessment: UNFOUNDED.

This statistic, though cited by the UN, OECD, and other reputable organisations, appears to have no basis whatsoever. (Moreover, though many of these sources repeat ‘US$2.6 trillion, or 5% of global GDP’ as if these were equivalent, 5% of global GDP has not been US$2.6 trillion since 2006. Global GDP in 2019 was approximately US$87.8 trillion, 5% of which is US$4.4 trillion.) No organisation or advocate should cite this statistic under any circumstances.

Corruption statistic no. 4: Corruption, together with tax evasion and illicit financial flows, costs developing countries approximately US$1.26 trillion per year.

Who cites it: The claim that corruption, tax evasion, and illicit financial flows together cost developing countries approximately US$1.26 trillion per year has been advanced by the UN, as well as by several UN agencies such as UNDP and UNESCO. The statistic has also been cited by the European Union, the WEF Partnering Against Corruption Initiative, and Transparency International (TI).

Origin of this statistic: The UN materials do not cite a source for this figure. The WEF report cited a Transparency International UK webpage for corruption statistics.5c102541bbb4 That webpage was no longer available at the time this Issue was written. However, correspondents at TI confirmed by email that the webpage listed the 2011 Global Financial Integrity (GFI) report Illicit Financial Flows from Developing Countries over the Decade Ending 2009 as its source for the US$1.26 trillion statistic. GFI, a civil society organisation, has released several reports that attempt to estimate the amounts that developing countries lose every year to illegal financial flows. (More recent TI statements, including a statement from the managing director, cite the UN as the source but do not include additional information.) Given this, we conclude that the origin of this statistic is GFI’s estimate of illicit financial flows in 2008, which appears in the report identified by our TI contact.

Our assessment: PROBLEMATIC.

GFI’s method for estimating illicit financial flows uses discrepancies in official trade and macroeconomic statistics to detect hidden financial flows. This method, though widely used and considered reliable by many, has also been subject to criticisms. For example, an OECD report notes that some academic critics have argued that GFI’s estimation method makes unrealistic assumptions about things like transport costs, and neglects other factors that could account for discrepancies in international trade and finance statistics. We do not, however, view these criticisms, on their own, as sufficient to render the GFI estimates of illicit flows non-credible.

That said, even if we put aside critiques of GFI’s methodology, there are other problems with relying on GFI for the blanket declaration that corruption, tax evasion, and illicit financial flows cost developing countries US$1.26 trillion per year. First, the 2008 figure was on the high end for the years (2000–2008) that this report examined: the average annual illicit outflows during this period were between US$725 billion and US$810 billion. More recent GFI reports also indicate rather different estimates. A 2014 GFI report estimates the amount lost by developing and emerging economies in illicit financial flows over 2003–2012 at US$6.6 trillion, or an average of US$660 billion per year, while a 2017 report puts the average annual illicit outflows in 2014 at between US$620 billion and US$970 billion. There’s no obvious reason to use the 2008 figure when more recent data from the same organisation is available.

Second, and more importantly, GFI purports to estimate illicit financial flows from all sources, not just (or even mainly) from corruption. It would therefore be inaccurate to rely on the GFI data to state (as the title of a WEF piece does) that ‘corruption costs developing countries US$1.26 trillion every year’, or (as a UNESCO resource guide puts it) that ‘corruption, bribery, theft and tax evasion cost some US$1.26 trillion to developing countries each year’, even if the US$1.26 trillion figure were an accurate statement of average illicit outflows from developing countries. Since this is the way the statistic appears to be most often presented in the corruption literature, we find its use problematic.

If the GFI estimates of illicit financial flows were cited only as evidence for the magnitude of illicit flows overall – for example, if an organisation said something like, ‘In the 2000s, illicit financial flows from developing countries, including but not limited to the proceeds of corruption and other illegal activities, were estimated at roughly US$660 billion per year’ – then we would assess the statistic as credible, notwithstanding legitimate concerns about GFI’s methodology. Those wishing to use GFI’s estimates for illicit flows are further advised to make use of more recent GFI estimates.

Corruption statistic no. 5: Approximately 10%–25% of government procurement spending is lost to corruption each year.

Who cites it: The claim that 10%–25% of government procurement spending is lost annually to corruption has been advanced by the World Bank, UNODC, and the OECD, as well as by numerous civil society bodies including Transparency International and others.

Origin of this statistic: The organisations that cite this figure either cite no source or rely on one or more of three documents: (a) Transparency International’s 2006 Handbook for Curbing Corruption in Public Procurement; (b) UNODC’s 2013 Guidebook on Anti-Corruption in Public Procurement and the Management of Public Finances; and (c) a 2013 OECD guide, Implementing the OECD Principles for Integrity in Public Procurement. However, none of these documents provides any explanation for how this statistic was estimated, nor do they refer to other sources for the original calculations. The UNODC guidebook declares that ‘various studies suggest that an average of 10-25 per cent of a public contract’s value may be lost to corruption’, but it does not provide citations to any of these studies. Similarly, the OECD guide asserts that ‘in public procurement, studies suggest that up to 20-25% of the public contracts’ value may be lost to corruption’, but it does not identify any such studies. And the earliest of the three sources, the 2006 TI handbook, simply declares, in its opening paragraph, that ‘damage from corruption [in public procurement] is estimated at normally between 10% and 25%, and in some cases as high as 40 to 50%, of the contract value’; it does not cite a source for these numbers.

We have been in touch by email with people involved in the production of all three of these sources. The person whom we contacted regarding the UNODC guidebook stood by the estimate, but was unsure of the original source. We got more specific information from our correspondence with people involved in producing the OECD guide and the TI handbook. With respect to the OECD guide, to the best of our correspondents’ recollection, the estimate was based on the 2006 TI handbook, as well as national estimates cited by select OECD member countries in an expert group on integrity and public procurement. As for the TI handbook, our correspondents reported that the statistic in that handbook was a ‘summary’ of estimates presented in various case studies, surveys, and publications, as well as publicly available data on bribery convictions, though these sources do not appear to have been described or cited in the handbook itself. One person we contacted who was involved with the production of the TI handbook recalled that the sources included the World Bank’s Governance at a Glance research.

After conducting our own review, we have identified, as potentially relevant publications, two World Bank working papers by Charles Kenny, from 2006 and 2007. The 2006 paper cites, among others, a 2005 working paper by Nathaniel Hobbs, whose interviews with World Bank staff suggest that bribes in World Bank–financed projects are ‘usually between 10-15% of the value of the contract’ and that ‘in sub-Saharan Africa at least, a sum of 10-15% of contract value is lost through corruption’. Kenny’s 2006 paper also includes a table from the Business Environment and Enterprise Performance Surveys (BEEPS), showing construction firms’ estimates of typical payoffs to secure a contract as a percentage of the contract value. Based on the survey findings, Kenny estimated that ‘the average perceived payoff for a government construction contract in the region is around 7 percent of the contract value’ (although we will see this number is uncertain). Of course, the size of bribes as a percentage of contract value is not the same thing as the amount of government procurement spending lost to corruption. (Indeed, while kickbacks may lead to cost overruns and inefficiency, bribes are a direct cost to the firm, not to the government.)

While we think the above email exchanges clarify the likely sources for this statistic, for completeness we note the possibility that the original source for this ‘10%–25% of procurement spending lost to corruption’ statistic might be the same Kaufmann 2005 paper that provided the ‘US$1 trillion in annual bribe payments’ statistic discussed earlier. Given that TI relied upon an unlisted range of data sources, and that we have yet to discover UNODC’s source, the Kaufmann paper may still provide a piece of the puzzle, and so we will briefly discuss this paper as another possible source for this statistic

In addition to estimating total annual bribe payments, Kaufmann and his team attempted to use a 2003 World Bank Institute online survey (apparently no longer publicly available)f64d7aaec5cb to estimate what they call the ‘worldwide bribe-fee commission in tainted procurement’. The description in the paper is rather vague, but it appears that this survey asked respondents (probably firm managers) to estimate the percentage of a public contract’s value that firms like theirs must pay as kickbacks to government officials. If the estimated range for procurement bribes is similar to the Kaufmann team’s estimate for total bribes generally – in the US$604 billion to US$1.76 trillion range – and if, as the Kaufmann piece asserts, total global procurement spending in 1998 was approximately US$5.5 trillion, and if we further assume no significant changes between 1998 and 2003, then that would produce an estimate of procurement bribes, as a percentage of total procurement spending, in the range of 11%–32%.

However, there are a number of difficulties with relying on such a calculation. First, as noted above, the size of procurement bribes is not the same as the amount of procurement spending lost to corruption. Second, even putting aside all the concerns mentioned earlier about the Kaufmann team’s method, recall that their estimate of the total amount of global bribery was in the range of US$604 billion to US$1.76 trillion. Procurement bribery would only make up a portion, and probably (though this is admittedly speculation) a relatively modest proportion, of total bribery. Third, the Kaufmann paper’s presentation of its methodology on this point is too opaque to rely on it for quantitative estimates of procurement bribery.

Our assessment: PROBLEMATIC.

While we tend to believe that procurement corruption is both widespread and costly, and we are open to the possibility that 10%–25% of public procurement spending might be lost to corruption, we were not able to identify a study that produced a transparent and reliable source for this widely cited estimate. Both the TI and OECD estimates turn out to be best guesses, drawn from a range of sources, including interviewees’ subjective perceptions (which do not seem to have been collected in any systematic way). UNODC’s reference to ‘various sources’ suggests that this is the case for the statistic in their guidebook as well. The more data-based estimates referenced in these documents seem to focus on estimating the procurement bribes as a percentage of contract size, not the percentage of government spending lost to corruption. It is also worth noting that the estimates, such as they are, all seem to derive from sources that are roughly 15 years old.

In light of all this, we would caution organisations against citing this estimate without significant qualifications. That said, it might be appropriate to note something along the lines of, ‘International development officers working in the early 2000s conjectured that roughly 10%–15% of public procurement spending was lost to corruption’, as it does appear that this is an accurate representation of the original source material. Additionally, there are a number of more focused studies, including audits of individual programmes in various countries, that could be used to offer concrete and quantified examples of the extent of procurement fraud in some settings, and this might be preferable to relying on outdated and subjective conjectures to produce a global estimate.

Corruption statistic no. 6: Approximately 10%–30% of the value of publicly funded infrastructure is lost to corruption each year.

Who cites it: The assertion that 10%–30% of the value of public spending on infrastructure is lost to corruption every year appears in publications from the World Bank, IMF, and OECD.

Origin of this statistic: Many of the organisations that cite this statistic rely, directly or indirectly, on an October 2012 press release from the Construction Sector Transparency Initiative (CoST). However, it is not clear where CoST – now known as the Infrastructure Transparency Initiative – got that information.315a82d9e74a Subsequent statements from CoST have cited this statistic but have attributed it to Transparency International and the OECD. The OECD cites CoST’s 2012 press release as the source for this statistic, and so is unlikely to have been the original source. CoST’s reference to Transparency International, though, is a clue that TI, or some TI document, is likely the original source. Indeed, an official at CoST confirmed to us by email that TI’s 2005 Global Corruption Report, which focused on corruption in the construction industry, was the principal source for the CoST estimate.

However, the 2005 TI report does not conclude that, in general, 10%–30% of public infrastructure spending is lost to corruption. The report does reference a leaked internal World Bank document, prepared by the Bank’s Jakarta office in 1997, which found that most Indonesian government agencies had systems for the diversion of 10%–20% of the development budgets that they managed, and that in total roughly 20%–30% of Indonesian government development funds were diverted to public officials and politicians. It appears that organisations using TI’s 2005 report did not realise that these figures were (a) only for one country, Indonesia (at the tail end of President Suharto’s long and extremely corrupt regime), and (b) not about infrastructure exclusively. It also appears that the two statistics have been combined by others to generate a 10%–30% estimate for corruption-related losses. The TI report itself never uses this specific range.

Our contact at CoST also informed us that, in addition to relying on the TI report, CoST drew on data from the OECD and the American Society of Civil Engineers (ASCE) as presented in a 2013 DfID ‘How to Note’ by John Hawkins entitled Reducing Corruption in Infrastructure Sectors. This paper cites several sources. First, it cites TI’s 2005 Global Corruption Report. Second, it cites a 2009 journal article by Charles Kenny for the estimate that the global average cost of infrastructure corruption is 5%–20%. Kenny’s article, however, is not the original source for the 5%–20% estimate. Rather, Kenny’s piece cites as the source an apparently unpublished 2006 World Bank paper by Mohinder Gulati and M. Y. Rao, which focuses on the electricity sector. We believe that this working paper is likely an earlier version of a chapter published by Gulati and Rao in a 2007 edited volume. But while this chapter contains an extensive and informative discussion of the costs of corruption in the electricity sector, it does not contain the ‘10%–25% loss from corruption’ statistic.

We also came across a 2007 World Bank research paper by Kenny that includes, in a table discussing different kinds of construction-related corruption, a cell that states (without citation to any specific study or data source) that the ‘aggregate size of payments/theft’ attributable to ‘bidder collusion on price with connivance of government officials’ is ‘very large,‘ with ‘evidence of 10-30 percent mark-ups’. While a 10%–30% mark-up is different from a 10%–30% loss, one could imagine how this figure (which, again, is presented without citation or substantiation) could have been interpreted as evidence for the claim that 10%–30% of infrastructure spending is lost due to corruption.

The Hawkins paper also refers to estimates from the OECD and ASCE, but provides no further information, and neither organisation is included in the Hawkins paper’s bibliography. On further independent investigation, we did locate a 2008 journal article that cited a 2004 ASCE press release for the claim that corruption is responsible for an estimated US$340 billion in worldwide construction costs annually. That estimate, however, appears in the press release without citation to any source, and it is not expressed as a percentage of annual infrastructure spending. We also found, on the ASCE website, a policy statement (approved first in 2005 and again in 2020) which states, ‘Transparency International conservatively estimated 10% of the global infrastructure investment was lost through bribery, fraud, and corruption.’ The fact that ASCE relies on TI for this figure suggests that ASCE was not an original source for any attempt to quantify corruption in the infrastructure sector.

With respect to the Hawkins paper’s reference to the OECD as a possible source, we searched and found a 2005 OECD report that suggested that the cost of corruption ‘can be greater than 20% of the total original contract price’ in public contracts in areas such as construction. That estimate appears to be based on 2000–2003 World Bank Enterprise Surveys in six developing countries. Those surveys asked firm managers what proportion of the contract price firms like theirs need to pay in kickbacks in order to win a public construction contract. The responses ranged from 9.3% (Ghana) to 24% (Paraguay). While it seems like a bit of a stretch, it is possible that these figures (or perhaps similar data from other World Bank surveys) were later characterised as finding that roughly 10%–30% of public construction spending was lost to corruption, even though (a) the numbers do not quite match up, and (b) these estimates are not for public sector losses, but rather for the size of kickbacks paid by private sector firms. It is also possible that these surveys were used to corroborate the (misinterpreted) TI statistic.

Our contact at CoST also referred us to a more recent IMF publication, a 2015 staff report on public investment. The authors of the paper calculate efficiency scores using an output-oriented data envelopment analysis model and find ‘average inefficiencies in public investment processes of around 30 percent’. While the authors note that higher perceived corruption levels are associated with lower levels of public investment efficiency, they do not make a concrete statement about the scale of corruption in publicly funded infrastructure projects.

Our assessment: UNFOUNDED.

The estimate that 10%–30% of infrastructure spending is lost to corruption might be reasonably accurate for some countries. However, due to insufficient referencing, we have been unable to trace this statistic back to a clear source. Most of the trails we followed ultimately led back to TI’s 2005 Global Corruption Report. But that report does not actually purport to estimate the value of global infrastructure spending lost each year to corruption; rather, the relevant sections of the report describe the scale of corruption in Indonesia in the 1990s. Our best guess is that the ‘10%–30% of public infrastructure spending lost to corruption’ statistic is based on a mischaracterisation of the 2005 TI report. We therefore classify this statistic as unfounded.

Corruption statistic no. 7: Approximately 20%–40% of spending in the water sector is lost to corruption each year.

Who cites it: The assertion that 20%–40% (or, in some sources, 10%–30%) of the value of public spending in the water sector is lost to corruption every year appears in publications from UNDP, UNESCO, the German Development Agency (GIZ) and Transparency International.

Origin of this statistic: The original source for this estimate is somewhat obscure. Both the UNDP and GIZ reports, which use the 20%–40% figure, cite the Water Integrity Network’s 2016 Water Integrity Global Outlook (WIGO); this in turn references a ‘World Bank estimate’ but does not cite a World Bank document. The WIGO report does, however, cite a 2006 publication from the Swedish Water House, which again references ‘estimates by the World Bank’ without citing any specific World Bank paper or dataset. We are thus left to speculate which ‘estimates from the World Bank’ the Swedish Water House document might be referring to.

Our best guess is that the reference in the Swedish Water House report is either to unpublished data that is not publicly available (and therefore impossible to assess), or to a 2006 World Bank working paper by Janelle Plummer and Piers Cross entitled Tackling Corruption in the Water and Sanitation Sector in Africa. However, this paper does not purport to develop estimates of the ‘leakage’ rates for public resources devoted to this sector. In fact, when discussing the question of leakage rates in the African water and sanitation sector, the paper observed that ‘hypotheses on the scope and incidence of corruption [in this sector] are largely untested’; it added that while leakage ‘can be roughly estimated through comparative and limited sector studies’, such studies had not yet been performed in Africa.

The paper does cite a 2004 article by Jennifer Davis for an estimate of 20%–35% leakage rates in water and sanitation service delivery in South Asia. But the Plummer and Cross paper emphasises that ‘this estimate is limited to petty corruption and does not account for high level abuse or diversion of resources’. Furthermore, the Davis article does not purport to provide a quantitative estimate of leakage rates in the South Asian water and sanitation sector. Rather, the estimate noted by Plummer and Cross appears as a single mention in passing, where Davis writes, ‘Although it is not possible to estimate the full costs to the agencies (and, by extension, to the public) of corrupt behaviors in [water and sanitation] service delivery documented in our case studies, it is not unreasonable to suspect that these institutions regularly spend 20–35% more on construction contracts than the value of the services rendered.’

The only other place where the Plummer and Cross paper provides an estimate of leakage rates in the water sector – and, we suspect, the ultimate source for the 20%–40% statistic – is a footnote that says, in its entirety, ‘The limitation of the figures commonly quoted in the sector is a key concern addressed in this paper. These types of (20-40%) figures are often derived only from leakages in utilities, but they are actually similar to the levels of corruption quoted generally and in similar programs (such as rural development programs, roads projects) elsewhere.’

A member of the editorial team of the Water Integrity Network’s WIGO report, whom we contacted by email, confirmed that they also struggled to trace the ‘20%–40% of spending in the water sector is lost to corruption’ statistic back to a clear original source. This is why the WIGO report uses cautious and conditional language when presenting this estimate. Our contact also informed us that the Water Integrity Network is working to collect more reliable statistics on the scale and impact of corruption in the water and sanitation sector.

As noted above, some sources estimate corruption-related losses in the water sector at 10%–30% rather than 20%–40%. The 10%–30% estimate appears to trace back to Transparency International’s 2008 Global Corruption Report. The lead article in that report states, ‘Putting an exact financial cost on corruption is difficult. While a best-case scenario might suggest that 10 per cent is being siphoned off from the [water] sector annually in corrupt practices, a worst-case scenario places the figure at 30 per cent.’ However, no further explanation or reference is provided.

A later chapter of the 2008 TI report includes the following text: ‘In addition to the minimum corruption tax on contracts, the system described in South India also includes ‘savings on the ground’ from contractors delivering fewer or lower-quality products and services than mandated by their contracts, and when engineers sign off on poor performance. Such haggling can bring the total rake-off to 25–50 per cent.’ The source for this figure is Robert Wade’s 1982 article on irrigation projects in India. Wade reported, based on interviews with participants, that officials authorise overpayments to contractors at a rate of somewhere between 25% and 50% of the value of the irrigation projects that they work on, though the percentage may be lower on larger projects. This estimate, however, is almost 40 years old and is based on a single region in India.

Our assessment: UNFOUNDED.

While there are reasons to suppose that the amount lost to corruption in the water and sanitation sector might possibly be in the 10%–30% or 20%–40% range, we were unable to trace these quantitative estimates to anything other than educated guesses and assessments of certain individual projects. There is also no basis for attributing these quantitative estimates to the World Bank as an institution. So far as we can tell, two World Bank researchers published a paper that did not purport to estimate quantitatively the magnitude of corruption-related resource leakages in the water sector, but rather speculated that leakage rates might be in the range of 20%–35% or 20%–40%. The only study cited for this range was itself speculative. Both of these papers were also focused on specific geographic areas (Africa and South Asia, respectively), which may or may not be representative. The 1982 Wade article is similarly limited to one country and is therefore not an appropriate basis for drawing global conclusions.

We acknowledge that, under our evaluation criteria, we might have classified this statistic as problematic rather than unfounded, given that it does seem to trace back to a handful of genuine sources. In the end, we concluded that these sources discussed statistics that were sufficiently distinct from the estimate of 20%–40% of water sector spending lost to corruption. We therefore concluded that this latter statistic appears to be based more on unsubstantiated guesswork than on problematic extrapolation. But we acknowledge the role of subjectivity in these close cases, and if a reader thinks that the ‘problematic’ label would be more appropriate, we would not be inclined to contest the point.

Corruption statistic no. 8: Up to 30% of development aid is lost to fraud and corruption each year.

Who cites it: The claim that up to 30% of development aid is lost annually to fraud and corruption has been asserted most prominently by former UN Secretary-General Ban Ki-moon. It has also been repeated in publications from non-governmental organisations like Transparency International and the International Federation of Red Cross and Red Crescent Societies. A similar statistic, that between 20% and 40% of overseas development assistance (US$20 billion to US$40 billion) is stolen each year, has been cited by the European Parliament, Transparency International and U4, and the OECD.

Origin of this statistic: The source of the 30% statistic appears to be a speech that then Secretary-General Ban gave at a 2012 UN panel discussion. Every other source we could find that asserts that up to 30% of development aid is lost to corruption attributes this statistic, directly or indirectly, to Secretary-General Ban’s speech. Unfortunately, neither the secretary-general’s original remarks nor any of the subsequent commentary provides a source for the 30% estimate. (The related but distinct figure that 20%–40% of development assistance is stolen appears to originate in other sources, which we will discuss further below.)

We conjecture that the secretary-general may have been referring to corruption-related losses from the UN Global Fund to Fight AIDS, Tuberculosis and Malaria. This US$21.7 billion fund was created in 2002, and according to multiple reports, it has been plagued by serious corruption. An independent audit of the Global Fund, commissioned by the UN and reported in the media in early 2011, found that between 30% and 67% of the Global Fund’s money was misspent. It is possible that the secretary-general was referring to these audit results (referencing the low end of the range), and that he either misspoke (intending to refer only to the Global Fund) or was implicitly extrapolating the audit results for the Global Fund to all development aid. Again, though, this is only a guess.

Another possibility is that the secretary-general was relying on the 20%–40% statistic that appears in other sources, including the 2019 World Public Sector Report published by the UN Department of Economic and Social Affairs. The original source for this statistic seems to be a 2005 book by Raymond Baker entitled Capitalism’s Achilles Heel, along with a 2003 policy report by Baker and Brionne Dawson. (Some sources misattribute this data to the World Bank, likely because a reference to Baker for this statistic appears in a 2007 World Bank/UNODC document.)

However, Baker and Dawson did not purport to provide an estimate of the amount of development assistance stolen. Rather, they estimated the total amount of cross-border movement of corrupt money at US$20–US$40 billion per year, and they then compared this amount to total annual development assistance flowing into developing countries (which they estimated at about US$50 billion per year, though by 2003 that number was actually closer to US$70 billion). It is not clear where Baker and Dawson got the estimate that US$20–US$40 billion in corrupt proceeds flows across borders each year, and it is even less clear how later sources converted this into an estimate that US$20–US$40 billion of development assistance is stolen every year. Moreover, by Baker and Dawson’s own calculations, US$20–US$40 billion would not represent 20%–40% of development assistance, but rather 60%–80%. (More recent annual net aid flows are substantially higher, in excess of US$165 billion.) So far as we can tell, the claim that 20%–40% or US$20–US$40 billion of development aid is lost to corruption appears to be based on a misreading of an old and largely unsubstantiated statistic concerning a related but different matter.

Our assessment: PROBLEMATIC.

Were it not for the news reports on the UN Global Fund’s internal audits, we would have classified this statistic as unfounded, given the lack of any direct reference to specific studies or data. We do think it is likely, however, that the original source of the 30% statistic is the audit report on the Global Fund, and while the full audit report is not publicly available, we would be inclined to treat the media coverage of the auditor’s findings as credible. But even so, extrapolating from the fraud and corruption detected in this one programme to corruption-related losses in all development assistance seems problematic at best.

For one thing, Secretary-General Ban, and the sources that have cited his remarks, have stated that ‘up to’ 30% of development aid is lost to corruption, implying that 30% is the upper end of the estimated range. However, the audit results for the UN Global Fund indicate that 30%–67% of the fund’s money was misspent, in which case 30% would be the lower end of the range. For another thing, there is no particularly good reason to suppose that corruption-related loss rates for this one programme are broadly representative of loss rates for other programmes, or for development aid overall. The Global Fund was managed by the UN, operated in approximately 150 countries, and focused specifically on the health sector – and three diseases in particular. Other development aid programmes are targeted towards a different set of countries, are run by other donors (which may have quite different management systems and internal controls), and target a range of different sectors. Corruption-related losses in other programmes may be higher or lower than loss rates in the Global Fund, which itself represents only a tiny fraction of overall development aid. One might suppose that the fact that Secretary-General Ban’s estimate corresponds rather closely to the 20%–40% estimate found in other sources would increase the credibility of both estimates. But as we have seen, the 20%–40% estimate has even shakier foundations, relying as it does on an apparent misrepresentation or misunderstanding of numbers that are themselves more than 15 years out of date.

For these reasons, it is not appropriate to continue to cite the general statistic that approximately 30% of development aid (or 20%–40% of development aid) is lost to corruption, at least if the source for this figure traces to the secretary-general’s 2012 remarks or to Baker’s work from the early 2000s. The audit findings from the UN Global Fund may be useful for both advocacy and analysis, but those results can and should be cited more precisely – not as ‘up to 30% of all development aid is lost to corruption’, but rather, ‘an independent audit of projects sponsored by the UN Global Fund to Fight AIDS, Tuberculosis and Malaria found that 30%–67% of the funds were misspent, often due to corruption’.

Corruption statistic no. 9: Customs-related corruption costs World Customs Organization members at least US$2 billion per year.

Who cites it: The claim that World Customs Organization members lose at least US$2 billion per year from corruption has been cited by the OECD, the World Trade Organization (WTO), the UN Conference on Trade and Development (UNCTAD), and Transparency International.

Origin of this statistic: Transparency International cites the OECD as the source of this statistic, while UNCTAD cites the WTO. Both the OECD and the WTO appear to rely on a 2012 article by Bryane Michael, Frank Ferguson, and Alisher Karimov entitled Do Customs Trade Facilitation Programmes Help Reduce Customs-Related Corruption? However, while this article purports to use a combination of prior studies and statistical analysis to estimate the total government revenue losses attributable to corruption, and asserts confidently that ‘we know that corruption costs customs agencies world-wide about US$2 billion’, the statistical techniques and prior studies described in the paper are opaque, and there are serious questions about their validity. The authors describe their estimation technique as follows:

‘We first calculated from UN and World Bank data the estimated amount of customs duties collected by each country (in current US dollars). We then applied “penalties” to that income based on the findings of previous econometric studies. These previous studies found that four variables have a significant effect on customs-related corruption: overall level of corruption in the country, the average tariff rate, the extent of fraud in imports and the level of GDP. Based on these previous findings, we weighted to [sic] “penalty” of high corruption and tariffs more heavily than the other variables. Because we used officially declared customs revenues as the base of our calculations, we do not include the effect of lost taxes for concealed and un-declared imports. . . . We simply exploit [the] relationship emerging from data in previous statistical studies to derive our “best guesses” at losses related to corruption in the customs services of various countries.’

We confess that we have difficulty following this passage. We interpret it to mean that the authors took previous studies (not specifically identified) that found four variables that were correlated with corruption in customs, and then used the regression coefficients from those studies to estimate how much larger customs revenues would have been if corruption had been lower. But we are not sure. Moreover, the factors that the authors identify as correlating with customs corruption might also correlate with customs revenue through other channels. (The average tariff rate, for example, would likely have a direct association with customs revenue, independent of any association with customs corruption.) So, in the end, we do not find that this paper provides a satisfactory foundation for strong claims about the amount that trade-related customs costs governments each year in customs revenue.

Our assessment: PROBLEMATIC.

The estimate that customs corruption costs World Customs Organization members roughly US$2 billion annually comes from a single academic study. While this study was published in a reputable journal, the US$2 billion estimate of corruption-related customs losses was not the main focus of the study, and the statistical techniques used to generate this figure are not presented sufficiently clearly to assess the methodology. Moreover, the description that is provided suggests potentially serious flaws in the estimation technique. Additionally, as the authors of the study emphasise – but as sources citing the study typically do not mention – the estimates of customs-related revenue losses vary substantially across countries.

We have not been able to locate any more recent sources that provide reliable and transparent estimates of corruption-related customs revenue losses. We suggest that organisations that want to cite such an estimate should include, at the very least, appropriate caveats to acknowledge the uncertainty of the estimates and opacity of the techniques used to generate them. They should also avoid attributing these estimates to organisations like the OECD or WTO when in fact these organisations do not appear to have published their own analyses or to have attempted to independently assess the single academic study on which the most widely cited estimate is based.

Corruption statistic no. 10: Approximately 1.6% of annual deaths of children under 5 years of age (over 140,000 deaths per year) are due in part to corruption.

Who cites it: The claim that 1.6% of annual child deaths are due in part to corruption has been made in a joint publication from the World Health Organization (WHO) and UK Aid. It has also been cited by multiple civil society organizations, including Transparency International and the Global Organization of Parliamentarians Against Corruption (GOPAC). U4 publications have also used this statistic.

Origin of this statistic: The original source appears to be a 2011 academic article by Matthieu Hanf et al. entitled Corruption Kills: Estimating the Global Impact of Corruption on Children Deaths. The authors conducted a cross-country linear regression analysis using 2008 data for 178 countries, with the log of under-5 child mortality (deaths per 1,000 live births) as the outcome variable. The model included, as an explanatory variable, the country’s score on the 2008 Transparency International Corruption Perceptions Index (CPI). The model also included a number of control variables, including geographic and climate factors, other political variables (such as civil liberties), health expenditure, access to sanitation, per capita GDP, and demographic variables. The study found that the CPI (which gives countries higher scores if they are perceived as less corrupt) had a statistically significant negative correlation with child mortality.

To calculate the total number of child deaths attributable to corruption, it appears that the authors first used the estimated linear coefficient on the CPI variable to calculate the estimated child mortality rate for each country if the CPI were at its maximum possible value (10 out of 10, on the CPI scale used in 2008). They then took the difference between this child mortality estimate and the country’s actual child mortality rate for each country, and finally summed across all countries. This method yielded the widely cited result that child deaths would be lower by approximately 140,000 (a decrease of 1.6 percentage points) if there were no corruption.

Our assessment: PROBLEMATIC.

Although the Hanf et al. study is interesting and important, there are some clear difficulties with the extrapolations that the authors use to generate the estimate of 140,000 excess child deaths attributable to corruption.

First, as the authors acknowledge, there are a number of potentially important variables that they could not control for in their analysis.

Second, the calculations are based on treating the correlation coefficient for the CPI as a point estimate. However, even relatively small changes in that coefficient – well within the statistical confidence interval – could lead to vastly different estimates for the total number of child deaths associated with corruption. This study provides the point estimate but does not provide the estimates for the high and low ends of the standard 95% confidence interval.

Third, the analysis treats the CPI as if it were on a cardinal scale – that is, the difference between a 1 and a 2 is the same as the difference between a 2 and a 3, and between a 3 and a 4, and so forth. But there is no reason to believe that this is true. That is not too much of a problem when the objective is to figure out whether the CPI has a statistically significant correlation with the outcome variable of interest. But when the linear coefficient is used to quantify aggregate effects, the non-linearity of the CPI scale may create substantial problems.

Fourth, as the authors note, because several of the control variables may themselves affect, or be affected by, the CPI score (such as per capita GDP, per capita health expenditure, and access to sanitation), there are serious problems with using the partial correlation coefficient on the CPI variable to estimate the aggregate effect of corruption.

Perhaps for these reasons, the Hanf et al. paper is appropriately cautious about how it presents its estimates of the number of child deaths attributable to corruption. The authors write, for example, that based on their analysis, ‘it could be hypothesised that roughly 1.6% of world deaths in children could be explained by corruption’, which would imply that ‘more than 140000 annual children deaths could be indirectly attributed to corruption’ (emphasis added). However, many of the organisations that have cited (directly or indirectly) Hanf et al.’s work have not been as circumspect, suggesting a degree of certainty in this estimate that the authors themselves do not embrace.

Another illustration of a potentially misleading use of this source comes from an article jointly authored by the chair of the board of Transparency International and a project specialist at the WEF Partnering Against Corruption Initiative. They write, ‘Each year, US$7.35 trillion is spent on healthcare worldwide, but US$455 billion is lost to fraud and corruption, leading to the deaths of more than 140,000 children.’ This sentence clearly implies that it is the theft of health care funds that causes the deaths of 140,000 children. The source cited for this claim is a 2018 report from the National Academy of Sciences, Engineering, and Medicine, entitled Crossing the Global Quality Chasm: Improving Health Care Worldwide. That report presents a number of ‘key findings’ including these two: (a) ‘Approximately US$455 billion of the US$7.35 trillion spent on health care annually worldwide is lost each year to fraud and corruption’, and (b) ‘Globally, 1.6 percent of annual deaths in children under 5 – more than 140,000 deaths – can be explained in part by corruption.’ The Hanf et al. article is the source for the second claim. Note, however, that the National Academy report does not attribute the child death statistic exclusively to the estimated health spending fraud; these are two separate findings, with two separate sources. The Hanf et al. study reports an association between child mortality and perceived corruption generally, rather than an association specifically with health spending lost due to corruption.

Of the ten corruption statistics we covered in this Issue, this was the one that we came closest to deeming credible. And if the statistic were presented in more general terms – ‘Researchers have found strong evidence that corruption increases child mortality rates’, with a citation to the Hanf et al. paper – we would indeed rate the claim as credible, notwithstanding the fact that the Hanf et al. finding, like most such findings, can be criticised on a variety of grounds. But the form in which the claim most often appears in prominent statements and publications from international organisations suggests far more precision and certainty than is warranted, and we find this use of the Hanf et al. result problematic.

Summary of findings

Table 1 summarises our assessments and conclusions regarding the ten corruption statistics reviewed above.

Discussion

Our findings are disappointing. When we began this project, we anticipated that at least a few widely cited global corruption statistics would turn out to be unfounded, but we also expected that we would identify some examples of corruption statistics that we could endorse as credible. Indeed, we hoped that we could use these examples as positive illustrations of ‘best practices’. However, of the ten corruption statistics we assessed, not a single one could be classified as credible. Only two (an estimate of illicit financial outflows, and an estimate of corruption’s impact on child mortality) came close. Six of the ten statistics are problematic (some seriously so), and the other four are, as far as we can tell, unfounded.

We do not believe that this is because we applied unrealistically high standards. We are well aware that public reports, briefing papers, speeches, and advocacy documents are not academic papers, and that some simplification – including of quantitative statistics – is inevitable. We also acknowledge that developing reliable global corruption estimates is not easy. As we stressed in the introduction to this Issue, our criteria for classifying a statistic as ‘credible’ did not require an absence of concerns about the methodology or the data. But there is a difference between statistics that are imperfect simplifications and statistics that are based on guesswork, wildly misleading extrapolations from mostly unrelated data, or nothing at all.

So what explains the fact that all ten of the statistics we investigated – statistics that are either problematic or entirely unfounded – have been cited in public statements, reports, and speeches from leading organisations and their senior leaders? We do not know for sure, but we have a few conjectures.

First, it is possible that, when it comes right down to it, the accuracy of the corruption statistics that these organisations cite is not terribly important to either the organisations or their target audiences. Perhaps the main reason these organisations cite quantitative corruption statistics is to underscore the point that corruption is widespread and harmful. To accomplish that goal, any large-sounding number will do. (For a similar point with regard to illicit financial flows, see Forstater 2017.) Considered in that light, it may not matter much whether (for example) the estimate of annual global bribery is US$500 billion or US$1 trillion or US$5 trillion, because these all sound like big numbers.

If that’s the case, though, why do organisations feel the need to cite descriptive quantitative statistics at all? Why not instead cite the large research literature establishing that corruption is associated with a range of bad outcomes, together with vivid individual cases? Here again, we can only guess. Perhaps the explanation is that quantitative statistics add (unearned) rhetorical power or a (misleading) veneer of scientific certainty. Or perhaps the culture of many organisations is infused with the idea that if something cannot be measured quantitatively then it isn’t important, and as a result, those who want to elevate the priority given to corruption feel the need to quantify the problem, using whatever numbers are readily at hand.

Another possible explanation for the persistence of problematic and unfounded corruption statistics is that those working in this field depend too much on two misleading proxies for reliability. First, the association of some of these statistics with reputable organisations may give what turn out to be casual guesstimates the appearance of scientific rigour. Thus, a speculative conjecture about the magnitude of some form of corruption, expressed by a World Bank researcher in an unpublished working paper, becomes a ‘World Bank estimate’. Second, the very fact that certain statistics are repeated over and over, in multiple sources, may make these statistics appear to reflect a consensus view among experts, when in fact all these different sources are citing each other, or are all relying (without realising it) on the same unreliable original source.

While this is disheartening, we acknowledge that the adverse consequences of relying on problematic or unfounded statistics in this context may not be terribly serious. As noted above, it seems that these statistics are often used as decoration – as a kind of rhetorical flourish, to convey the general idea that corruption is very big and very bad – but are not meant to do any serious work in formulating anti-corruption strategies or evaluating policies. If so, then perhaps the prevalence and persistence of unreliable statistics in anti-corruption discourse is mostly harmless. After all, the message that they are meant to convey – that corruption is widespread and harmful – is almost certainly accurate.

Still, influential international organisations, government agencies, and civil society groups – including U4 – can and should do better. The widespread citation of unreliable corruption statistics is inimical to efforts to better understand the nature of the problem, and may unhelpfully obscure areas of uncertainty that require more attention and more research. (We also underscore that these unreliable corruption statistics might actually understate the extent and harms of corruption, and to that extent might disserve, at least to some degree, the rhetorical purpose for which these statistics are sometimes cited.) Also, as we noted in the introduction, the anti-corruption community is increasingly embracing and promoting evidence-based policymaking and evidence-based advocacy. Both as a matter of principle and in the interest of maintaining credibility, the organisations pushing for evidence-based anti-corruption strategies should be more careful about the quality of the evidence that they present.

Towards better corruption statistics

We conclude this Issue by offering five suggestions for improving the use of corruption statistics (or, for that matter, any quantitative statistics) in public-facing reports and statements.

1. Always trace back to (and, in written documents, cite and/or link to) the original source. Before citing a quantitative statistic in a public document or speech, one should always trace the statistic back to its origin. Sometimes a source will be cited in document A, but document A got the source from document B, which got it from C, etc. Always try to locate the original source for the statistic in question and attribute it to that source, not to some intermediate source that cites the statistic (possibly inaccurately). If it is impossible to locate the original source, the statistic should not be cited. If the original source says something vague like ‘Studies have shown that…’, without actually referencing a specific study, the statistic should not be cited. If the original source says something like ‘According to World Bank estimates…’, but does not reference a specific World Bank document or data set, the statistic should not be cited.
2. Read the original source carefully. A non-specialist need not scrutinise the source the way an academic might. Given the inherent difficulty in measuring hidden activities like corruption, all estimation techniques will be open to questions and criticisms. And sometimes the original source will be based on non-public data, making independent assessment impossible. Notwithstanding these important caveats, before an organisation or one of its officials cites a corruption statistic in a public document or speech, someone in the organisation should read the original source carefully to make sure that the approach to estimation is basically understandable and sensible. At the very least, someone must verify that the quantitative statistic is actually based on some sort of quantitative analysis and is not simply a guess expressed in quantitative form. It is also important to confirm that that the original source is estimating the same quantity that the statistic purports to measure – to make sure, for example, that an estimate of the total cost of corruption to the global economy is not actually from a source that is estimating the total amount of global money laundering, or that an estimate of the amount of public procurement spending lost to corruption is not actually from a source that estimates the size of the kickbacks that private contractors pay to public officials. Rigorous academic evaluation is not obligatory, but basic due diligence is.
3. Do not conflate an author’s institutional affiliation with the institution’s official findings. As noted above, the credibility of certain statistics is artificially enhanced when they are presented as if they were the official estimates of a reputable institution, like the World Bank or IMF, when in fact the statistics in question were produced by someone employed by (or consulting for) that organisation. When relying on estimates in, for example, a World Bank working paper, one should say, ‘A World Bank working paper estimated…’ rather than ‘the World Bank estimated…’, unless it is clear that the document in question represents the organisation’s official findings.
4. Do not exaggerate certainty, precision, or generality. Simplification is necessary in an advocacy or policy context, but oversimplification is a problem. Often the original source for a corruption statistic will be limited to a certain time, country, region, or sector. Additionally, the original source will often acknowledge considerable uncertainty about the estimate (or the uncertainty will be obvious, even if it goes unacknowledged). As statistics are repeated from source to source, these important caveats tend to drop away, creating a misleading impression of a precise number that can be generalised to a broad (often global) context. This can and should be avoided by briefly noting the limits on the scope of the statistic and acknowledging the uncertainty. Doing so might lead to fewer ‘global’ statistics of the sort we investigated in this Issue. We might not be able to say, with any reasonable degree of confidence, what percentage of infrastructure spending is lost to corruption each year. But we could perhaps find several evocative examples of specific countries or programmes where a rigorous evaluation produced a more reliable estimate of corruption-related loss rates in those programmes. Those individual examples can be just as powerful in making the rhetorical point about corruption’s destructive effects.
5. Avoid ‘decorative’ statistics and focus instead on evidence of significant effects or associations. This is perhaps our broadest and potentially most controversial recommendation. We suggest that international organisations, donor agencies, civil society groups, and others reconsider their penchant for ‘decorative’ quantitative statistics. Rather than peppering reports and speeches with large-sounding numbers and percentages, we suggest that these influential organisations focus on empirical evidence of statistically and substantively significant correlations between corruption and other variables of interest, especially when those correlations can be plausibly interpreted as reflecting a causal relationship. Rather than trying to quantify, for example, the amount that corruption costs the global economy each year (in absolute or percentage terms), a report or speech that wants to make the point that corruption has a significant adverse economic impact could cite the extensive research literature finding that corruption is associated with lower per capita incomes, higher inequality, and more frequent macroeconomic crises. We might not be able to say, with any reasonable degree of certainty, how many annual child deaths are due to corruption, but we can cite numerous statistical studies (including the Hanf et al. paper discussed earlier) as support for the proposition that there is a strong correlation between corruption and child deaths, as well as a range of other adverse health outcomes. Shifting the focus from (unreliable) global descriptive statistics to empirical evidence of causal effects would also effect a productive shift in the discourse from general descriptions of the problem to consideration of consequences and causes.