Reading the 2019 Election Polls
The 2019 General Election is likely to be one of the most difficult to estimate in recent history.
At the London School of Economics, its Department of Methodology hosted an event on how to read the opinion polls for the upcoming General Election.
Polling stations open at 7am on Thursday 12th December, and postal voters have already received their ballots.
Prof. Patrick Sturgis (LSE), who led the inquiry into the 2015 polling miss, began by recounting the differing natures of errors in polls in the past two general elections.
In 2015, the two main parties appeared tied. A systematic overestimation of Labour vote intention share — coupled with a similarly-sized underestimation of the Conservative share — meant the impression of a close race was given. All final published polls from British Polling Council members estimated ties or leads for either party of under two points.
In 2017, there was a wide spread of Conservative leads in the final polls, ranging from one point (with Survation’s telephone poll) to 13 points (BMG Research’s mixed-mode sample).
Even though the polls correctly averaged on the Conservative lead, Labour were substantially underestimated. The direction of travel throughout the campaign was well-tracked, even if the levels of each party was missed.
As Prof Sturgis said:
Even if the levels are wrong, the polls are good at detecting changes.
Prof Sir John Curtice (University of Strathclyde) reminded us that opinion polling is a fragile and difficult exercise. Weighting made no net difference in the 2015 polls, which appeared to demonstrate the risk of availability bias. Those who answer surveys quickly may differ from those who take longer.
- Very little weighting or modelling data to imitate demographic variation in turnout at the last election;
- Wider use of either requiring voters to choose from among candidates standing in their constituency or reallocating first preferences.
Following removal of attempts to replicate the pattern of turnout at previous elections, reported statistics are now closer to each company’s unweighted data than in 2017. Indeed, it is tending to have the opposite effect than in 2015.
Polls appear less likely to have a deficit of younger voters. There is some tendency for weighted recall votes to be more Conservative than the actual vote, but this may reflect compensation for false recall. Prof Curtice asked at the end:
Are the polls potentially vulnerable to underestimating the differences as happened in 2015?
Prof Sara Hobolt (LSE) began:
Is this a Brexit election?
The UK’s exit from the European Union is considered by voters to the most important issue facing Britain today. With an electorate split roughly evenly over Brexit, identities have emerged — which are stronger than traditional party identification.
Some alignment along these Brexit lines has already occurred, with clear differences in policy preferences of different party and referendum groups.
Brexit has emerged as a salient political identity — partly capturing a cultural divide. Party competence and leader perception matter too.
Joel Williams (Kantar) described his company’s approach to estimating vote intentions.
Kantar’s Lifepoints panel is used, meaning voting questions are part of a standard market research survey — where only those who explicitly agree to answer questions on voting take part.
In order to deal with the missing non-voters problem, Kantar runs a model of voting likelihood for each respondent, based on their stated likelihood to vote, age, whether they voted last time and registration. This voting likelihood model is then used in Kantar’s weightings.
Among those with a stated preference, the Conservatives had a 43% share and Labour were on 30%. Undecided respondents are then ‘squeezed’ — meaning a follow-up question is asked. People who gave a first preference to a party that is not standing in their constituency were reallocated. People who still said they didn’t know had their preferences imputed. Williams highlighted that a uniform likelihood-to-vote weighting (as in, everyone turns out) was “never going to happen”.
Kantar are currently building a mixed-mode panel with respondents drawn by random sampling, for future political research.
Joe Twyman (Deltapoll) said the new political opinion constituency were running internet panel polls of national vote intention for the Mail on Sunday. In addition, they were also conducting some constituency polls by telephone.
Twyman talked about the differing choices for Remain-leaning Conservative supporters and Leave-leaning Labour supporters. The presentation also described emotional resonance scoring, and how awareness is not agreement, which is not emotional resonance, which is not behavioural change.
Anthony Wells (YouGov) begin his presentation with a comparison between the company’s traditional polling and the MRP estimates in 2017. A late ad hoc reallocation of Don’t Know respondents may be expressly avoided this time.
Wells talked about how YouGov’s samples are found. YouGov runs a UK panel of around 1.6m people (though there are fewer active panellists). When potential respondents open their emails, the survey they are going to answer is not yet decided — people are assigned, based on their demographics, to the survey most ‘in need’.
Sampling quotas are employed for their vote intention polls: age by gender by education, 2017 past vote (collected in 2017) by region, levels of political attention, social grade, and 2016 EU referendum past vote (again, collected in 2016). Questions are asked using constituency-based prompts, and the sample is weighted by the same criteria as the quotas.
Dr Jack Blumenau (UCL) and Prof Ben Lauderdale (UCL) gave their presentation on multi-level regression with post-stratification, seeking to translate votes into seats beyond uniform swings.
Turnout is less important than people think it is, and how votes switch between elections matters a lot. Switching is contextual, not proportional.
Using 100,319 YouGov respondents (159 per constituency) giving vote intentions between 19th–26th November 2019, a model of individual vote intention is built, based on a person’s demographics, which can vary by constituency.
The key individual-level predictors were 2016 referendum vote, 2017 General Election vote, 2019 European Parliament vote, age, gender, education, ethnicity, marital status, plus some key interactions (such as Scotland and past vote). The constituency-level predictors were region, functions of the 2016 EU referendum and 2017 General Election votes, first and second party ranking in 2017, candidate incumbency, defectors and parties standing down, and many economic and demographic variables.
The headline results were redacted during the talk, since they were to be published at 10pm that night.
Prof Jouni Kuha (LSE) presented the broadcaster’s exit poll. The BBC, ITV and Sky pay for Ipsos MORI to collect votes cast throughout the day, from 140–150 polling stations. Systematic samples of voters are asked how they voted as they leave the polling station, with mock ballot papers.
The secret to the exit poll is that it uses a ‘panel design’ — meaning researchers go back to the same polling stations. The change between exit polls is then used to model change in vote shares between elections.
Regression models are built for these changes, and then applied to all constituencies. These predicted vote shares and then converted into victory probabilities. These probabilities are then summed, giving the estimated number of constituencies for each party.
Will it be alright on the night? The methodology is proven, and mostly unchanged. The team is experienced, and mostly unchanged.
All will be revealed at 10pm on Thursday 12th December.