Numbers, Letters, and Weapons

The Office for Statistics Regulation sent two letters about claims by two politicians.

This article looks at these claims about poverty and COVID-19 prevalence. Some of these claims were inaccurate, in different ways.

The Prime Minister on Poverty Statistics

  • “Absolute poverty and relative poverty have both declined under this government.”
  • “There are 100,000 fewer children in absolute poverty.”
  • “There are…500,000 fewer children falling below low-income and material deprivation.”
  • “There are hundreds of thousands — I think 400,000 — fewer families living in poverty now than there were in 2010.”

The Department of Work and Pensions publishes several poverty measures. The official figures are the Households Below Average Income statistics. The latest report is for 2018/19.

These statistics derive from the Family Resources Survey. As survey estimates, there is a plausible range around each figure. The true value could be somewhat higher or lower.

After housing costs, the estimated level of relative poverty was 14.5m in 2018/19 (13.6m — 15.2m). This is the number of individuals in households with an income below 60% of that year’s median income.

This is higher than in the estimate in 2009/10 (13.6m). There are issues with comparisons before 2012/13, due to the use of 2011 census factors. It is also higher than in 2014/15 (13.5m). That holds before housing costs too. This claim is inaccurate for relative poverty.

The second claim holds, after housing costs. The estimated number of children in relative poverty was 2.5m in 2009/10. In 2018/19, the central estimate was 2.4m. That change could be due to sampling variation. The DWP estimate the same change for absolute poverty too.

We round the DWP figures, as these are estimates. (Image: BBC)

There were new questions for measuring material deprivation in 2010/11. Estimated number of children in low-income households with material deprivation was 1.7m.

By 2018/19, that estimate was 1.5m. The Prime Minister’s claim was inaccurate.

It remains unclear where the claim about “fewer families in poverty” comes from. The HBAI statistics are for individuals: both levels and proportions.

Both Full Fact and the Office for the Children’s Commissioner in England could not verify this claim.

Ed Humpherson’s letter agreed the Prime Minister made inaccurate claims:

Our team has investigated the statements which you highlight (and has reached the same conclusion that these statements are incorrect).

The problems of measuring poverty

No single figure about poverty tells the whole story so context is really important when drawing comparisons of poverty over time.

The four common measures of poverty used in the HBAI statistics are:

  • Relative poverty: households with under 60% of the median income in that year.
  • Absolute poverty: households with under 60% of the median income in 2010/11. This threshold is adjusted for inflation.
  • Housing costs: we can calculate both measures before and after housing costs.

Also, the DWP produces statistics on self-reported material deprivation.

These are the DWP figures over time. (Image: OSR)

The difficulty with many measures is that politicians and campaigners can select which ones they highlight. It can be unclear which measure of poverty someone means.

These choices can then shift, out of political convenience. The different measures should build our evidence of UK poverty.

Scotland’s First Minister on COVID-19 prevalence

For example, we assess that the prevalence of the virus in Scotland, right now, is five times lower than it is in England.

The basis for this claim was unclear.

As Ed Humpherson (OSR) identifies, the claim used two sources:

The CMMID figure was not published at the time the First Minister made their claim. Nor did the figure refer to England: it referred to the United Kingdom.

These are two different modelling exercises. Even in the same family of models, different choices can lead to different estimates. A direct comparison is difficult.

The model estimates are uncertain. The latest CMMID estimate for the UK was 0.45%. There is a range around this figure: 0.24% — 0.92%.

As the CMMID figure was unpublished, a third source was used:

  • ONS COVID-19 Infection pilot survey: this estimate is for English private households. The survey estimates the number of people who would test positive for SARS-CoV-2.
This modelling is not comparable to the headline estimate either. (Image: ONS)

This survey does not estimate the prevalence of SARS-CoV-2 in England. That would require additional assumptions about accuracy of the tests. The survey is only English private households. The survey excludes hospitals, care homes, and other institutional places. The report states:

We do not report on the prevalence rate within the analysis sections of this bulletin.

There is uncertainty in that estimate too. The headline estimate was 51,000 people in English private households would test positive for SARS-CoV-2. That figure is for 8th — 21st June. The confidence interval ranged from 21,000 to 105,000.

As Ed Humpherson writes:

We do not think that the sources above allow for a quantified and uncaveated comparison of the kind that was made. In future if such comparisons are made, we would expect to see sources made publicly available and a clear explanation of the limitations and associated uncertainty.

Numbers “are wielded like weapons”

In the political arena, data serves a different purpose: attack and defence. Prof Denise Lievesley (Oxford) said:

Numbers have a spurious credibility and are wielded like weapons.

The misuse of statistics is not limited to one political tribe. Public statistics should have proper sources, clear communication, and prompt corrections. Accuracy matters.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store