2019 Constituency Polls Accuracy

How accurate were constituency polls in the 2019 UK Election?

Before the 2019 UK General Election, research companies conducted over 30 constituency polls.

This article seeks to understand the methods of constituency polling, and their accuracy. The analysis finds a decline in mean absolute error as the election approaches. This is in line with national polling.

Methods

The British Polling Council said 2019 was “more accurate” than any election since 2005. This was for national polling. This analysis will look at constituency polling.

A constituency poll must meet the following criteria:

  • Target population: The target population must be one constituency.
  • British Polling Council: The company must be a British Polling Council member.
  • Fieldwork date: The survey fieldwork end-date must be after 1st October 2019.

This analysis does not look at constituency projections from national polls. Nor does it include MRP estimates.

Deltapoll

Deltapoll collected its responses by telephone, from landline and mobile numbers. The company calibrated responses to the profile of adults (18+) in each constituency. The weights were for gender, age, ethnicity, ward, a 2017 past vote recall target, and the 2016 EU referendum.

Deltapoll calculate weight targets by averaging actual and recalled 2017 vote shares. EU referendum recall targets come from areal interpolation estimated calculated by Prof Hanretty. Intention share estimates use self-reported likelihood to vote is on a 11-point scale (0 to 10) as a factor. These vote intention estimates exclude those who say they are 0 out of 10. Headline estimates exclude undecided responses. The question is:

In the upcoming general election, the following parties are standing in your constituency. If the general election were tomorrow, which candidate would you vote for?

Survation

Survation collected telephone responses, from a mix of landline and mobile numbers. The company weighted responses by six targets. These were: age, sex, ward, 2017 Election vote, 2016 EU referendum vote, and 2019 EP vote.

Self-reported likelihood to vote on a 11-point scale was then used to weight responses. Deltapoll and Survation use the exact same question wording. These headline estimates exclude undecided responses.

The aim

Show the error: the difference between the survey estimate and the actual result. This will remove parties that are not standing. On each graph, depict error bounds of five points.

Analysis

We look at the period between 75 days before the election and polling day. There were 32 constituency polls by Deltapoll and Survation.

Generally, survey errors reduced as we got closer to the election. Survey errors can come from real changes in vote intentions after the survey. This is often called late swing.

Labour were generally underestimated. (Image: RPubs)

One cause of such changes will be the availability of parties to vote for. Prospective parties may be on a polling company’s list of potential options. Those parties may not submit a nominated candidate. People who said they were intending to vote for a withdrawing party have to switch or not vote.

The largest recorded error was on Survation’s poll of Portsmouth South. The survey covered 28–29 October 2019, interviewing 406 people by telephone. The central estimate was that Labour were on 24%, six points behind the Liberal Democrats (on 30%). The Brexit Party were estimated to be on 14%. ‘Some other party’ had a published vote intention share of 6%.

At the election, Labour won 49% of the vote in Portsmouth South. The Liberal Democrats received 11%. The three party absolute error for that poll was about 18 points. That is the average total survey error for the Conservatives, Labour, and the Liberal Democrats.

Major changes in vote intention can happen during elections. A poll can be accurate when it was conducted, yet show a large error by election day. This error can come from smaller parties withdrawing or not campaigning. Survation’s poll was before candidates were known.

Labour held Portsmouth South. (Image: BBC)

Deltapoll conducted a later poll of Portsmouth South. The 500 telephone interviews were on 22–27 November. The poll’s vote intention estimate for Labour was 45%. The corresponding confidence interval approximately runs from 41% to 49%. The Liberal Democrat estimate was 11%, matching the actual result. The three party absolute error for this poll was 1.6 points.

Differences between the two polls may be in part due to house effects.

For polls ending after 21st November, the mean three party absolute error was 3.7 points. The average error on the Labour share was -2.9 points. The negative sign means polls generally underestimated the party.

Average absolute errors were around four points. (Image: RPubs)

Limitations

Beaconsfield was not shown on the absolute error graph. There was no Liberal Democrat candidate in that seat.

I chose the bound of five points based on a simple random samples and the size of the respective polls. This is simplistic. Such a bound is too wide for smaller parties. Sampling error is narrower at smaller proportions. Non-sampling errors matter for total survey error.

There are design factors which lead to wider confidence intervals.

Party vote shares may be misunderstood in analysis as being somehow independent. If one party is being over-estimated, then by definition, another is under-estimated. The same criticism holds when considering vote shares over time. For one party to rise, others must fall.

Conclusion

From October 2019, there were 32 constituency polls. Of these, 14 were undertaken in the final three weeks of the election campaign. In these surveys, 12 out of 14 polls ‘called’ the candidate who won the seat in the General Election. 13 had suggested the correct top two parties.

These later polls had a mean three party absolute error centred on around four points. Survation had conducted two of these 14 surveys.

Surveys provide estimates, subject to many sources of potential error. Constituency polling is a fraught and difficult exercise. The accuracy in 2019 is notable, given the interest in close contests.

R code and survey data for the graphs is available on GitHub. Survey estimates come from the Deltapoll and Survation online archives. There is an R Pubs document to view.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store