Numbers, Letters, and Weapons

The Office for Statistics Regulation sent two letters about claims by two politicians.

This article looks at these claims about poverty and COVID-19 prevalence. Some of these claims were inaccurate, in different ways.

The Prime Minister on Poverty Statistics

In recent months, the Prime Minister claimed:

  • “Absolute poverty and relative poverty have both declined under this government.”

The Department of Work and Pensions publishes several poverty measures. The official figures are the Households Below Average Income statistics. The latest report is for 2018/19.

These statistics derive from the Family Resources Survey. As survey estimates, there is a plausible range around each figure. The true value could be somewhat higher or lower.

After housing costs, the estimated level of relative poverty was 14.5m in 2018/19 (13.6m — 15.2m). This is the number of individuals in households with an income below 60% of that year’s median income.

This is higher than in the estimate in 2009/10 (13.6m). There are issues with comparisons before 2012/13, due to the use of 2011 census factors. It is also higher than in 2014/15 (13.5m). That holds before housing costs too. This claim is inaccurate for relative poverty.

The second claim holds, after housing costs. The estimated number of children in relative poverty was 2.5m in 2009/10. In 2018/19, the central estimate was 2.4m. That change could be due to sampling variation. The DWP estimate the same change for absolute poverty too.

Image for post
Image for post
We round the DWP figures, as these are estimates. (Image: BBC)

There were new questions for measuring material deprivation in 2010/11. Estimated number of children in low-income households with material deprivation was 1.7m.

By 2018/19, that estimate was 1.5m. The Prime Minister’s claim was inaccurate.

It remains unclear where the claim about “fewer families in poverty” comes from. The HBAI statistics are for individuals: both levels and proportions.

Both Full Fact and the Office for the Children’s Commissioner in England could not verify this claim.

Ed Humpherson’s letter agreed the Prime Minister made inaccurate claims:

Our team has investigated the statements which you highlight (and has reached the same conclusion that these statements are incorrect).

The problems of measuring poverty

There is no universal definition of poverty. Different measures capture different things. There are different concepts too, such as income and adequate food. As Elise Baseley (OSR) writes:

No single figure about poverty tells the whole story so context is really important when drawing comparisons of poverty over time.

The four common measures of poverty used in the HBAI statistics are:

  • Relative poverty: households with under 60% of the median income in that year.

Also, the DWP produces statistics on self-reported material deprivation.

Image for post
Image for post
These are the DWP figures over time. (Image: OSR)

The difficulty with many measures is that politicians and campaigners can select which ones they highlight. It can be unclear which measure of poverty someone means.

These choices can then shift, out of political convenience. The different measures should build our evidence of UK poverty.

Scotland’s First Minister on COVID-19 prevalence

The First Minister in Scotland claimed:

For example, we assess that the prevalence of the virus in Scotland, right now, is five times lower than it is in England.

The basis for this claim was unclear.

As Ed Humpherson (OSR) identifies, the claim used two sources:

The CMMID figure was not published at the time the First Minister made their claim. Nor did the figure refer to England: it referred to the United Kingdom.

These are two different modelling exercises. Even in the same family of models, different choices can lead to different estimates. A direct comparison is difficult.

The model estimates are uncertain. The latest CMMID estimate for the UK was 0.45%. There is a range around this figure: 0.24% — 0.92%.

As the CMMID figure was unpublished, a third source was used:

  • ONS COVID-19 Infection pilot survey: this estimate is for English private households. The survey estimates the number of people who would test positive for SARS-CoV-2.
Image for post
Image for post
This modelling is not comparable to the headline estimate either. (Image: ONS)

This survey does not estimate the prevalence of SARS-CoV-2 in England. That would require additional assumptions about accuracy of the tests. The survey is only English private households. The survey excludes hospitals, care homes, and other institutional places. The report states:

We do not report on the prevalence rate within the analysis sections of this bulletin.

There is uncertainty in that estimate too. The headline estimate was 51,000 people in English private households would test positive for SARS-CoV-2. That figure is for 8th — 21st June. The confidence interval ranged from 21,000 to 105,000.

As Ed Humpherson writes:

We do not think that the sources above allow for a quantified and uncaveated comparison of the kind that was made. In future if such comparisons are made, we would expect to see sources made publicly available and a clear explanation of the limitations and associated uncertainty.

Numbers “are wielded like weapons”

Statistics is the science of uncertainty. We want to understand data, to build evidence and improve decisions.

In the political arena, data serves a different purpose: attack and defence. Prof Denise Lievesley (Oxford) said:

Numbers have a spurious credibility and are wielded like weapons.

The misuse of statistics is not limited to one political tribe. Public statistics should have proper sources, clear communication, and prompt corrections. Accuracy matters.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store