Exit Poll Performance, 2005–2019

How has the broadcasters’ exit poll performed in the last five elections?

In the television coverage, the first major moment of the night is the seat estimates from the broadcaster’s exit poll.

The article considers the accuracy of the exit poll in the last five elections, and how uncertainty might be shown.

Performance

In the 2019 General Election, the exit poll was conducted by Ipsos MORI, and paid for by the BBC, ITV and Sky News. The fieldworkers try to go back to the same polling stations as they did in the last election. People coming out of the polling station are asked to fill out a duplicate ballot paper, replicating the vote they just made. 144 polling stations are currently part of this panel, and Ipsos MORI received 19,607 duplicate ballots on 12th December.

Analysed by a team of psephologists and statisticians, the change from the previous exit poll to the latest one is used to build a model for change between elections. This is known as the Curtice-Firth methodology of exit polling — used to overcome the major difficulty in the lack of counts by polling stations.

This exercise is all done under immense time pressure, as the exit poll results are broadcast at 10pm.

Here are the results of the 2019 General Election exit poll (compared to the actual result¹):

Conservative: 368 (365)
Labour: 191 (203)
Scottish National Party: 55 (48)
Liberal Democrats: 13 (11)

Image for post
Image for post
The exit poll was, once again, accurate. (Image: Ipsos MORI)

This exit poll was again accurate, with an error of three seats for the Conservatives and 12 seats for Labour.

We can look at errors in the past five elections for the Conservatives, Labour and the Liberal Democrats (with 2001 also shown):

Image for post
Image for post
The largest error for a single party was 15 seats. (Image: R Pubs/ggplot2)

Whilst the 2015 General Election is sometimes highlighted as the broadcasters’ exit poll failing to make the right ‘call’, the error on the Conservative seat estimate was just 15. This is the largest error for a single party in the last five General Elections.

The difference between a party barely failing to get an overall majority and having a slim majority may be small — but its political ramifications are large.

Uncertainty

Uncertainty is inherent in any estimation exercise.

In the exit poll for the 2017 General Election, the credible interval for the Conservatives seat was 297–330 seats. The central estimate was 314 seats — which was a little lower than the final total of 318.

The short summary statement contains a lot of information. For example, ‘Conservatives largest party’ indicates:

  • The credible interval for Conservative seats encompasses both above and below 326. It is not certain whether the Conservatives have a majority.
  • The credible interval for the Conservative and Labour difference in seats is entirely above 0. The analysts do not plausibly believe the Labour seat count is larger than the Conservatives.

At present, these credible intervals are not shown to the viewer. Putting the credible intervals alongside the central estimates may be one solution. Another would be an extended summary, explaining in two or three sentences what the exit poll estimates mean.

Spending more time with the chief analyst (Prof Curtice on the BBC, Prof Rallings on ITV, Prof Thrasher on Sky) after the exit poll announcement would be welcome, to help guide the viewer and subsequent discussions.

¹In the BBC election results, the Speaker is allocated against their original party. The R Code for the graph may be read on R Pubs, and the data table is published online.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store