Fact-Checking Stats of the Year

It may be a prominent statistic, but is it right?

I was invited to investigate various submissions to the Royal Statistical Society’s Statistic of the Year panel. Alongside my fellow Statistical Ambassadors (Graham Wheeler, Rhian Davies and Lucy Teece), we found data sources and checked claims.

This article looks at ratings and reasoning for the stats I checked — because the stats life chose me.

Image for post
Image for post

Nomination: This summer in the UK was the hottest summer since records began, beating 1976 by 0.2 degrees Celsius.

Rating: Partially true

Reasoning: The provisional mean temperature in England in June — August 2018 (Summer 2018) was 17.16 degrees Celsius, compared to 17.01 in 1976 — the previous record.

The Times article is referring to England, and not the UK as a whole. For the UK, the mean temperature was 15.80 degrees in 2018 — barely beating the previous record of 15.78 in 2006.

Nomination: 14 million people in food crisis in Yemen.

Rating: False

Reasoning: This is referring to a worst-case scenario:

Nomination: 1 in 200 people in UK are homeless.

Rating: Needs clarification

Reasoning: Shelter’s central estimate combines people living in temporary accommodation, rough sleepers, people in single homeless hostels, and people in social services TA.

The main component (accounting for 295,000) is based on statutory homelessness: it is an estimate for people, as the MHCLG data is for households. Using ONS mid-year population estimates: 1 in 201 people in Great Britain are homeless.

Nomination: 32.6% of children in the UK are in poverty.

Rating: Needs clarification

Reasoning: It is an experimental measure created by the Social Metrics Commission, based on living in a household below 55% of the median of total resources available (three-year rolling average).

This is different to using absolute or relative household income and housing costs, or indices of material deprivation. These are the measures that feature in the DWP’s Households Below Average Income report (which are National Statistics, and based on the ONS Family Resources Survey). Hence, this statistic needs clarification.

Nomination: DEFRA cars that are diesel: 98%.

Rating: Needs clarification

Reasoning: It was reported that, in late 2017, 98% of the DEFRA fleet (of about 4,000 cars) were diesel. The leases undergo a four-year cycle, and there appears to be no public record of the fleet’s composition.

The earliest report I can find is within an interview by Fleet News with Dale Enyon:

A parliamentary written question in November 2017 revealed DEFRA had 2 electric vehicles and 9 ultra-low emission vehicles.

Nomination: 7.52: UK happiness level (out of 10).

Rating: True

Reasoning: The figure being quoted is for April 2017 to March 2018. People are asked by the Office for National Statistics how happy they felt yesterday, on a scale of 0 to 10.

The mean score out of 10 for that period was 7.52.

Image for post
Image for post

The estimate was 7.54 for the latest available period (July 2017 to June 2018).

The question is part of the Annual Population Survey: we should highlight that the happiness score is a survey estimate.

Nomination: Proportion of Under-25's that don't drink any alcohol: 1/3rd.

Rating: Partially true

Reasoning: This comes from the Health Survey for England, conducted annually by NatCen. The academic paper underlying the stories states: “Among those aged 16 to 24 years, the proportion of non-drinkers increased from 18% (95% CI 16–22%) in 2005 to 29% in 2015 (CI 25–33%)”.

This claim refers only to young adults in England (in 2015), is based on self-reporting, and the proportion is more like 3 in 10.

Nomination: Weekly amputations due to diabetes in the UK: 120 limbs per week.

Rating: Partially true

Reasoning: The claim is referring to ‘minor’ lower-limb amputation procedures due to diabetes in England (amputated below the ankle), for patients aged 17 or over.

The count of such amputations between 2014/15 and 2016/17 was 19,073. That is 122 per week (or 120 per week, rounded). There were major amputations due to diabetes too: 7,305 in the same time period.

Nomination: Measles Cases in Europe in 2018: 41,000.

Rating: Partially true

Reasoning: According to The Economist, the claim refers to the first six months of 2018.

In the WHO European region, there were 43,010 reported cases. The WHO receives data from its member states, which may explain the discrepancy (if earlier estimates were updated).

Image for post
Image for post

Nomination: 600% increase in betting adverts in UK in last 5 years.

Rating: False

Reasoning: This claim is based on Ofcom research, and refers to the increased estimated volume of commercial spots for gambling between 2007 and 2012 (234,000 to 1,389,000). This is not referring to the last 5 years.

Nomination: 99.6% of all TV viewers in Iceland watched their first game of the 2018 FIFA World Cup.

Rating: True

Reasoning: The Icelandic broadcaster RÚV stated in its report that [translated]: “The RÚV’s share during the game was 99.6%”.

It should be clear that TV audience share is not the same as the population proportion watching the game.

Nomination: 6,420 false or misleading claims made by Trump in 649 days.

Rating: Needs clarification

Reasoning: The Washington Post tracks and categorises claims made by President Trump. Between 20th January 2018 and 30th October 2018, this categorisation yielded 6,420 false or misleading claims in 649 days.

However, there are some limitations, and will require the statement to be clarified:

  1. The Washington Post database may be incomplete;
  2. The database counts repeated instances of the same claim separately;
  3. It is partially subjective (which is why the UK Statistics Authority prefers phrasings like ‘likely to be misleading’).

Nomination: 1 in 303 million, the likelihood of winning this year’s USA Mega Millions lottery.

Rating: True

Reasoning: This is the basic calculation behind matching all numbers with 5 balls (from 70), and one ‘Mega Ball’ (from 25):

Image for post
Image for post

(Also, I gave that probability to the BBC.)

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store