Pluralities and Suspending Parliament

An article in The Telegraph claims that: “54 per cent of British adults think Parliament should be prorogued to prevent MPs stopping a no-deal Brexit.”

This article looks at the survey behind that claim.

Question Wording

ComRes conducted a survey of 2,011 GB adults via an internet panel, between 9th and 11th August 2019. This survey was sponsored by The Telegraph.

The article’s headline was:

The polling question is worded somewhat differently:

Image for post
Image for post
Learning to read data tables is valuable. (Image: ComRes)

This question wording is more elaborate than a simpler statement about support for suspending or ‘proroguing’ Parliament.

This statement about delivering the policy of exiting the EU — and the lengths (“by any means”) that the Prime Minister “needs to deliver” that policy. “Suspending parliament if necessary” (that is, in extremis) is not the same thing as thinking that prorogation “should” be done.

Exclusion of Don’t Knows

In The Telegraph article, and in the poll’s press release, it is asserted that ‘more than half of the public’ or ‘a majority of British adults’ support this suspension.

The survey results showed 44% of the sample agreed with the statement, 37% disagreed and 19% did not know. 74% of 2016 Leave voters agreed with the statement, whilst 68% of Remain voters disagreed.

44% is a relative majority — a plurality — but is not “more than half of the public”. 54% is the agreed proportion once people selecting ‘Don’t Know’ are excluded.

Practice varies, but what needs to be explicitly clear to the reader is whether non-substantive responses are included or excluded. Claiming “a majority” support something when only a plurality does so is false.

Acquiescence Bias

We also have to cautious when reading results of agree-disagree questions.

Image for post
Image for post
Do you agree or disagree: we should avoid acquiescence bias? (Video: Pew Research Center)

There is a well-known problem in survey research called acquiescence bias. This is when respondents say ‘yes’ or ‘agree’ to questions, in order to be agreeable or to just get through the survey. This tendency can cause major problems in interpretation, particularly when majorities agree with conflicting statements.

Another agree-disagree statement question given in the survey was:

For this statement, 39% agreed, 37% disagreed and 25% said they did not know. This question indicates the public suffers from greater ambivalence that the question about delivering the EU exit “by any means” might imply, albeit one asking about a hypothetical scenario.

In conclusion

Order effects were also suggested, but the statement order was randomised by ComRes.

Reporting on polls, particularly on aspects of public opinion that are hard to measure, is challenging. Additionally, question design is a difficult art.

It would be helpful to the public if uncertainty and limitations were effectively conveyed.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store