Our Survey Says: January 2019
Following from my article on misuses, misinterpretations, and misunderstandings of survey data in November 2018, I look at similar errors in January 2019.
In short
The ERSC Members project: This poll of Labour members induced numerous errors, including a Lord not including the fact it was of Labour members.
Inaccuracy on the BBC: On Question Time, Diane Abbott MP correctly stated that two main parties were “kind of level-pegging”, but the host wrongly stated Labour were “definitely behind”.
Don’t Knows included
George Eaton, Deputy Editor for the New Statesmen, wrote:
47% in the UK would not accept a Muslim as a member of their family.
The story is right, but the figure is incorrect. Mr Eaton erroneously assumes that non-affirmative answers (such as ‘Don’t Know’) have been excluded.
Whilst it remains the second highest rate in Western Europe, 36% of UK respondents — not 47% — said they would not be willing to accept a Muslim as a member of their family. 1,841 UK adults were interviewed by GfK via telephone, in April to August 2017.
Wrong Question, Wrong Population
Lord Adonis, a Labour peer, claimed on Twitter that:
Latest YouGov poll — 72% support a people’s vote
Conducted for the ESRC Party Members Project, the YouGov poll itself was of Labour members, rather than the general public. Moreover, the question was not about a personal desire for another EU referendum: it was about Labour policy. The question was:
Do you think Labour should or should not fully support holding a new referendum on Brexit?
The ESRC Party Members Project also funded a survey of the general public. Under the condition that Parliament has rejected the proposed exit deal, there is a marginal lead supporting another referendum (with 41% supporting, and 38% opposing).
Incomparable questions
Labour activist Dr Éoin Clarke, who writes on Twitter as ‘Tory Fibs’, asserted:
Support for a 2nd Referendum has dropped 14%
This is incorrect. Dr Clarke is comparing a question about Labour party policy (as Lord Adonis cited) to a different polling question.
The People’s Vote campaign funded a poll of Labour party members, conducted by YouGov via their internet panel between 13th and 18th September 2018. The question was:
When the negotiations with the European Union about Brexit are complete, would you support or oppose a public vote on the outcome?
This is about supporting “a public vote on the outcome” of completed negotiations, not whether Labour should “fully support” holding a new referendum.
Not a shock
The YouGov poll of Labour party members, funded by the ERSC Party Members Project, generated numerous errors.
One strange example was the following article in the Daily Mirror, which called this survey a ‘shock’ in its headlines:
Brexit: Vast majority of Labour members back second referendum reveals shock poll
As previously noted, the People’s Vote campaign poll of Labour members in September 2018 also found a high desire for holding a “public vote”. Additionally, that survey also estimated 93% of Labour members would vote Remain (after excluding those who say they don’t know).
About two-thirds of 2016 Remain voters want another EU referendum, irrespective of wording. This poll result should not have been a shock.
Questioning time
On the BBC’s Question Time programme, the following exchange took place between Isabel Oakeshott (author), Diane Abbott MP (Labour, Hackney), and Fiona Bruce (chair) [which has been edited for clarity]:
Oakeshott: You [Labour] are way behind in the polls. What makes you think you are going to win? You and your leader are miles behind the Tories!
Abbott: As a point of information, in the polls, we are currently kind of level-pegging.
Oakeshott: You are six points behind.
Bruce: You are behind, Diane.
Ms Abbott MP is correct. On 15th January, looking at Britain Elects’ rolling average of the most recent polls, Labour have a vote intention share of 38.3%, and the Conservatives have a share of 37.9%.
Ms Oakeshott appears to be referencing some recent polling by YouGov, and not a polling average.
We should be cautious about declaring which party is ahead. Accuracy in vote intention polls can only be determined after a general election. There are house effects: excluding YouGov, the latest poll prior to 15th January to show a Conservative lead was by Kantar in mid-November 2018. Kantar’s series had since shown a marginal Labour lead.
Not a push poll
Dr Aaron Bastani, founder of Novara Media, has taken to calling public polling commissioned by partisan groups “push polls”:
That is not a push poll. YouGov is a member of the British Polling Council and the Market Research Society.
AAPOR provide the following definition:
“Push polls” are actually political telemarketing — telephone calls disguised as research that aim to persuade large numbers of voters and affect election outcomes, rather than measure opinions.
In addition, Bastani’s posts refer to sampling via internet panels as “very weak” and not “methodologically robust” — despite its usage in market and social research all over the world:
As an example, there were 163 polls by British Polling Council members asking the EU referendum question, between 1st September 2015 and 22nd June 2016. 117 of these polls were conducted via internet panels.
