General Election 2019 Stat Checks: The Calm
The next UK general election will be held on 12th December 2019. During political campaigns, politicians and campaigners may misunderstand or misuse statistics — usually in pursuit of partisan arguments.
Statistics are important in public debate. Accuracy matters.
Throughout this election campaign, I will seek to check as many of these statistical claims as I can. Each claim will be checked, and given a rating with reasoning.
I will use multiple ratings throughout this article series — to assess statistical claims. Previously, I only needed the first four categories, as I was assessing entries to the Statistic of the Year. Two more categories are added:
- True: the claim accurately presents a statistic (or statistics), correctly described with relevant context.
- Partially true: the statistical claim would be true if reasonably amended, such as eliding between Great Britain and the United Kingdom.
- Needs clarification: the claim is vaguely stated or poorly-worded, and requires greater clarity to properly assess.
- False: The claim either wrongly describes a statistic or has false figures.
- Likely to be misleading: Statistics are presented in a manner likely to mislead the reader. Strange axes, small print, and other oddities go here.
- Implausible (about the future): We cannot fact-check the future. We can judge if a claim about the future is implausible.
Close in Chippenham?
Claim: “Opinion polls are now predicting a close win for the Liberal Democrats in the Chippenham constituency”.
Rating: Needs clarification.
Reasoning: Most public opinion polls measure current national vote intentions, across either Great Britain or the United Kingdom (including Northern Ireland). These polls do not make predictions — but sketch what public opinion looks like when the surveys were asked.
A uniform swing from national opinion polls conducted prior to the 1st October would suggest:
- the Conservative vote share to fall by 10 points;
- the Liberal Democrat share to increase by 13 points.
Apply these figures to Chippenham in 2017 yields a Conservative share of 45 points, and a Liberal Democrat share of 39 points.
What model is being used to translate national vote intention shares onto individual constituencies? Is that model reliable? Which parties are being assumed to stand in this ‘prediction’? What time is being referred to as “now” — given the leaflet was sent before the election date was set?
Claim: According to “YouGov polls for the Isle of Wight in the last six months, it’s the Conservative or Greens here”.
Reasoning: There is no such polling series. As is the case in Chippenham, this was a projection of national polling — but described falsely as constituency polling.
No British polling company surveys a single constituency with such frequency, nor are internet panels well-equipped to directly survey such an area.
Candidates are kindly reminded that opinion polling and projections based on that polling are different. That extra step of analysis has additional assumptions and further uncertainty. It should also be made clear to voters what projection you are using.
“57.57 per cent”
Claim: “57.57 per cent” intend to vote SNP in the General Election.
Rating: Needs clarification.
Reasoning: The “exclusive poll” was seemingly a Google Survey (a survey-wall) of Herald Scotland website visitors. Methodological issues with social research via survey-walls are not discussed in the Herald Scotland article.
The central estimates for SNP vote intention share from the latest opinion poll was 39% (Panelbase, 9th — 11th October). YouGov’s SNP share in an earlier poll (30th August — 3rd October) was somewhat higher, at 43%, but this difference may be due to house effects. These large differences with survey estimates from reputable polling companies are not discussed in the article.
It is rather silly to report to two decimal places (“57.57 per cent”). No reasonably-sized survey could be that accurate.
The Small Print
Claim: A poll of North East Somerset residents says the Liberal Democrats are on 32%, compared to 38% for the Conservatives.
Rating: Likely to be misleading.
Reasoning: As the small print states, the hypothetical vote intention question is based on an expected “very close” contest between the Conservatives and Liberal Democrats.
The question wording was:
Imagine that the result in your constituency was expected to be very close between the Conservative candidate and the Liberal Democrat candidate, and none of the other parties were competitive. In this scenario, which party would you vote for?
Hypothetical bias means we should be wary of treating these figures too literally. It simply suggests there is some potential for further tactical voting.
The constituency question — which used candidate names — estimated the Conservatives were on 44%, and the Liberal Democrats were on 28%. This was a Survation telephony poll of 405 adults in North East Somerset, conducted on 16–18th October 2019.
“Over 10 trillion pounds”
Claim: “A Corbyn government would see over 10 trillion pounds leave the country overnight.”
Rating: Implausible (about the future)
Reasoning: Allison Pearson, author and columnist for The Telegraph, made this assertion on Twitter.
According to the latest waves of the ONS Wealth and Assets survey:
Aggregate total net wealth of all households in Great Britain was £12.7 trillion in July 2014 to June 2016, up 17% from the July 2012 to June 2014 figure of £10.9 trillion.
From these survey estimates, Ms Pearson’s claim is not plausible.
The “ENORMOUS” mandate
Claim: A map of the United Kingdom shows large colours of ‘Leave’ showing an “enormous” mandate.
Rating: Likely to be misleading.
Reasoning: In 2016, Leave had 51.9% of 33.6m valid votes cast in the referendum on the principle of European Union membership.
This is not “enormous” — but a victory margin of 3.8 points.
The map itself intends to represent constituency estimates, rather than counting areas.
Showing a binary outcome by counting area is also likely to be misleading. Cartograms reflecting differing vote shares and populations are more effective at showing the 51.9% share that Leave obtained in 2016.
A cost of extension
Claim: Extending the UK’s membership of the European Union costs £1bn a month.
Reasoning: The Prime Minister has claimed:
It is a week since this parliament voted, yet again, to force Brussels to keep this country in the European Union for at least another three months, at a cost of £1 billion a month.
In 2018, the UK’s net contribution to the EU after public sector receipts was £11.0bn.
The alternate proposal to extending the UK’s membership is to enact the withdrawal agreement. That withdrawal agreement contains a transition period, lasting until December 2020. During that transition period, the UK government makes payments to the EU as if it were an EU member.
Consequently, there is no additional cost to these extensions.
One model to rule them all
Claim: “MRP analysis… is the only model that correctly predicted the shock wins for Labour in 2017”.
Rating: Needs clarification
Reasoning: Naomi Smith, Chief Executive of Best for Britain, claimed:
Our tool uses MRP analysis, which is the only model that correctly predicted the shock wins for Labour in 2017, like Canterbury and Kensington.
Multi-level regression with post-stratification is a technique for producing a model — not a model itself. What demographic variables were used in building these vote intention models? Which were the most important? Which companies were used for collecting the survey data? When was the fieldwork done?
Whilst people remember the performance of YouGov’s constituency estimates, the Lord Ashcroft estimates are somewhat forgotten. Indeed, YouGov’s model in 2015 had the same outcome as their standard polling — erroneously suggesting Labour and the Conservatives were tied.
Producing a model using the MRP technique does not automatically bestow accuracy. All that glitters is not gold.