Total False Recall

No, a recent Opinium poll was not based a 10 point Conservative lead in 2017.

It has been claimed that a recent Opinium poll suggested there was a 10 point Conservative lead in 2017, meaning their sample was not representative.

It did not. This article also discusses the problem of false recall.

What the poll said

Opinium conducted a survey of 2,003 UK adults, on 20th — 22nd November 2019. This poll was part of Opinium’s regular survey series for The Observer. The central vote intention estimates were the Conservatives with 47%, Labour on 28%, and the Liberal Democrats on 12%.

Opinium also produced a report, discussing various results from their polls. (Image: Opinium)

In a post shared on tax campaigner Richard Murphy’s blog (written by a contributor), it was claimed that this Opinium poll was “wrong”. In that poll, one question (Q:V005a) asks:

And thinking back to the UK general election in June 2017 , which, if any, of the following parties did you vote for?

46% of respondents selected Conservative, and 36% ticked Labour. Since this is a 10 points lead — when the actual result was a 2.4 point Conservative lead — is “over represented by 7.6%”. Consequently, the author writes the poll is a “magic trick”, “balls” and “wrong”. The co-founder of Novara Media also posted similar criticisms.

This criticism is ill-founded. Like YouGov, Opinium asks how people voted in a General Election soon after the event — using the first available recall in their weightings, rather than subsequent recall questions.

For clarity, this is the first vote recall from either June 2017, when the panellist joined the panel, or the latest survey.

What that table demonstrates is that some people do not correctly remember how they vote:

Outside of London, 11% of 2017 Labour voters could not correctly recall how they voted.

The false recall conundrum

To demonstrate the problem of using past vote recall in weightings, we can look at an illustrative and fictional past election: 50% backed Red, 40% voted Blue and 10% Green.

This is an illustrative example — to show the issue. (Image: R Pubs)

Now, imagine that 1 in 5 Red voters wrongly believe they voted Blue in the past election. This is called false recall or ‘differential recall error’.

In this election, every voter is backing the same party they believe they voted for last time. Imagine we draw a perfect sample: 50% backed Blue (including the 10% of voters who falsely recall voting Blue last time), 40% voted Red, and 10% Green.

50% of voters are backing Blue in this fictional election. (Image: R Pubs)

Our naive survey researcher thinks there are too many Blue voters in this sample, and too few Reds. They weight by the past vote.

Since no-one believes they have switched, current vote intention now looks exactly the past election. False recall has induced a polling error in our illustrative example:

Surely, no-one forgets how they vote? Unfortunately, they do. Some people may also misreport how they voted.

This can cause difficulty in social research, if you are asking what people did and felt years ago. There are signs this is happening in the UK, and may lead to errors in survey estimates.

False recall has been studied by academics too. The imperfection of memory is an established part of psychological research.

In trying to combat false recall, internet panel companies might treat their respondents like a longitudinal panel — taking the earliest available response about how they voted in elections and referendums. Alternately, some companies might elect not to weigh their responses by past vote recall.

Trying to account for false recall is not a ‘trick’. Opinion polling is a difficult exercise, and even seemingly ‘basic’ strategies may be fraught with trouble.

The R code for the graphs is available on R Pubs.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store