Clear Red Water

Polling of Labour members registered for LabourList is not “clear”.

The campaign group Momentum asserted “the polling is clear”:

The overwhelmingly majority of Labour members support the 2019 manifesto, while more members oppose the attack on [Jeremy Corbyn] than support it.

Survation’s polling of Labour members registered for LabourList was not “clear”. In that question, there was only a difference of around one point between the two options.

Previously in 2020…

In the 2020 leadership contest, Survation did two polls of LabourList registrants. These registrants also had to be members of the Labour party. These two polls were on 8th — 13th January and 21st — 24th February.

Survation’s poll of 1,196 LabourList registrants estimated Starmer’s support at 45%. YouGov’s polling of Labour members in their internet panel estimated that figure to be around 53%. YouGov’s survey of 1,323 Labour members was on 20th — 25th February.

For Starmer’s share, there was a difference of about eight points between the two companies. The voting period for the leadership election ended on Thursday 2nd April. There may have been some change in vote intentions after the polling.

Image for post
Image for post
Survation’s central estimate for Long-Bailey was 31%. (Image: Labour)

In the contest, Sir Keir Starmer’s share of first preferences was 56% — winning in the first round.

Running out of Momentum

On Twitter, the Momentum account claimed:

The polling is clear. The overwhelmingly majority of Labour members support the 2019 manifesto, while more members oppose the attack on [Jeremy Corbyn] than support it.

[Keir Starmer] are you listening? Because the Party is speaking.

The polling is not clear on this latter claim.

As Sienna Rodgers (LabourList) writes:

The results of the survey indicate that 48% of members believe Starmer was “wrong” not to restore the Labour whip to Corbyn following his reinstatement, while 46% think the move was “right” and 6% do not know.

Image for post
Image for post
That is not clear. (Image: Survation)

Survation surveyed 5,008 LabourList registrants who were also Labour members. The responses from email survey invites were between 23rd and 25th November 2020. The company weighed responses by age, sex, region, and the 2020 leadership contest vote. Survation are members of the British Polling Council, and abide by transparency rules.

The question was:

Which of the following statements best reflects your view?

  • 48% clicked “Keir Starmer was wrong to not restore the party whip to Jeremy Corbyn”.
  • 46% chose “Keir Starmer was right to not restore the party whip to Jeremy Corbyn”.

Surveys provide estimates, subject to many sources of potential error.

The margin of sampling error around a lead is larger than for a point estimate.

Image for post
Image for post
Leads have a bigger margin of sampling error. (Image: Pew Research Center)

The difference between the two estimates was around one point. That subtraction equal to two points (48% minus 46%) is due to rounding.

Assuming a simple random sample, the margin of sampling error for the lead is around three points. A plausible range for the difference between the two options was between -1 and +4 points.

In a survey of over 5,000 members, there were a difference of 65 respondents between “right” and “wrong”. Such a small difference could be due to sampling error alone. This is not “clear”.

There are other types of survey error too. The sampling frame is the LabourList email database. Those registrants may have higher average political engagement. There could be differences between members who register for LabourList and other members. That makes an inference to all Labour members difficult.

The choice of weights for a membership poll is also challenging. This is about how researchers count responses to reflect the population. How different groups comprise a party membership is rarely public knowledge. Survey researchers must make educated decisions about weighting targets. Despite their best efforts, those choices could be in error.

Survation’s polling was not “clear” that “more members oppose the attack”. The company asked a fair question to LabourList registrants. Respondents also had to be Labour members. For that question, the lead may be due to sampling error alone. LabourList registrants could differ from other members. That clouds any inferences about all Labour party members.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store