Fact-Checking Stats of the Year
It may be a prominent statistic, but is it right?
I was invited to investigate various submissions to the Royal Statistical Society’s Statistic of the Year panel. Alongside my fellow Statistical Ambassadors (Graham Wheeler, Rhian Davies and Lucy Teece), we found data sources and checked claims.
This article looks at ratings and reasoning for the stats I checked — because the stats life chose me.
Nomination: This summer in the UK was the hottest summer since records began, beating 1976 by 0.2 degrees Celsius.
Rating: Partially true
Reasoning: The provisional mean temperature in England in June — August 2018 (Summer 2018) was 17.16 degrees Celsius, compared to 17.01 in 1976 — the previous record.
The Times article is referring to England, and not the UK as a whole. For the UK, the mean temperature was 15.80 degrees in 2018 — barely beating the previous record of 15.78 in 2006.
Nomination: 14 million people in food crisis in Yemen.
Rating: False
Reasoning: This is referring to a worst-case scenario:
If current trends continue, an additional 3 million to 5.6 million Yemenis could become severely food insecure in the coming months, pushing the number of severely food insecure Yemenis up to 14 million in a worst-case scenario.
Nomination: 1 in 200 people in UK are homeless.
Rating: Needs clarification
Reasoning: Shelter’s central estimate combines people living in temporary accommodation, rough sleepers, people in single homeless hostels, and people in social services TA.
The main component (accounting for 295,000) is based on statutory homelessness: it is an estimate for people, as the MHCLG data is for households. Using ONS mid-year population estimates: 1 in 201 people in Great Britain are homeless.
Nomination: 32.6% of children in the UK are in poverty.
Rating: Needs clarification
Reasoning: It is an experimental measure created by the Social Metrics Commission, based on living in a household below 55% of the median of total resources available (three-year rolling average).
This is different to using absolute or relative household income and housing costs, or indices of material deprivation. These are the measures that feature in the DWP’s Households Below Average Income report (which are National Statistics, and based on the ONS Family Resources Survey). Hence, this statistic needs clarification.
Nomination: DEFRA cars that are diesel: 98%.
Rating: Needs clarification
Reasoning: It was reported that, in late 2017, 98% of the DEFRA fleet (of about 4,000 cars) were diesel. The leases undergo a four-year cycle, and there appears to be no public record of the fleet’s composition.
The earliest report I can find is within an interview by Fleet News with Dale Enyon:
The seven-year plan is essentially two replacement changes away for the four-year cycle car fleet, which numbers 4,000 and is currently 98% diesel.
A parliamentary written question in November 2017 revealed DEFRA had 2 electric vehicles and 9 ultra-low emission vehicles.
Nomination: 7.52: UK happiness level (out of 10).
Rating: True
Reasoning: The figure being quoted is for April 2017 to March 2018. People are asked by the Office for National Statistics how happy they felt yesterday, on a scale of 0 to 10.
The mean score out of 10 for that period was 7.52.
The estimate was 7.54 for the latest available period (July 2017 to June 2018).
The question is part of the Annual Population Survey: we should highlight that the happiness score is a survey estimate.
Nomination: Proportion of Under-25's that don't drink any alcohol: 1/3rd.
Rating: Partially true
Reasoning: This comes from the Health Survey for England, conducted annually by NatCen. The academic paper underlying the stories states: “Among those aged 16 to 24 years, the proportion of non-drinkers increased from 18% (95% CI 16–22%) in 2005 to 29% in 2015 (CI 25–33%)”.
This claim refers only to young adults in England (in 2015), is based on self-reporting, and the proportion is more like 3 in 10.
Nomination: Weekly amputations due to diabetes in the UK: 120 limbs per week.
Rating: Partially true
Reasoning: The claim is referring to ‘minor’ lower-limb amputation procedures due to diabetes in England (amputated below the ankle), for patients aged 17 or over.
The count of such amputations between 2014/15 and 2016/17 was 19,073. That is 122 per week (or 120 per week, rounded). There were major amputations due to diabetes too: 7,305 in the same time period.
Nomination: Measles Cases in Europe in 2018: 41,000.
Rating: Partially true
Reasoning: According to The Economist, the claim refers to the first six months of 2018.
In the WHO European region, there were 43,010 reported cases. The WHO receives data from its member states, which may explain the discrepancy (if earlier estimates were updated).
Nomination: 600% increase in betting adverts in UK in last 5 years.
Rating: False
Reasoning: This claim is based on Ofcom research, and refers to the increased estimated volume of commercial spots for gambling between 2007 and 2012 (234,000 to 1,389,000). This is not referring to the last 5 years.
Nomination: 99.6% of all TV viewers in Iceland watched their first game of the 2018 FIFA World Cup.
Rating: True
Reasoning: The Icelandic broadcaster RÚV stated in its report that [translated]: “The RÚV’s share during the game was 99.6%”.
It should be clear that TV audience share is not the same as the population proportion watching the game.
Nomination: 6,420 false or misleading claims made by Trump in 649 days.
Rating: Needs clarification
Reasoning: The Washington Post tracks and categorises claims made by President Trump. Between 20th January 2018 and 30th October 2018, this categorisation yielded 6,420 false or misleading claims in 649 days.
However, there are some limitations, and will require the statement to be clarified:
- The Washington Post database may be incomplete;
- The database counts repeated instances of the same claim separately;
- It is partially subjective (which is why the UK Statistics Authority prefers phrasings like ‘likely to be misleading’).
Nomination: 1 in 303 million, the likelihood of winning this year’s USA Mega Millions lottery.
Rating: True
Reasoning: This is the basic calculation behind matching all numbers with 5 balls (from 70), and one ‘Mega Ball’ (from 25):
(Also, I gave that probability to the BBC.)
