Pluralities and Suspending Parliament

An article in The Telegraph claims that: “54 per cent of British adults think Parliament should be prorogued to prevent MPs stopping a no-deal Brexit.”

This article looks at the survey behind that claim.

Question Wording

The article’s headline was:

Boris Johnson has public’s support to shut down Parliament to get Brexit over line, exclusive poll suggests

The polling question is worded somewhat differently:

Do you agree or disagree with the following statements?

Boris needs to deliver Brexit by any means, including suspending parliament if necessary, in order to prevent MPs from stopping it

Learning to read data tables is valuable. (Image: ComRes)

This question wording is more elaborate than a simpler statement about support for suspending or ‘proroguing’ Parliament.

This statement about delivering the policy of exiting the EU — and the lengths (“by any means”) that the Prime Minister “needs to deliver” that policy. “Suspending parliament if necessary” (that is, in extremis) is not the same thing as thinking that prorogation “should” be done.

Exclusion of Don’t Knows

The survey results showed 44% of the sample agreed with the statement, 37% disagreed and 19% did not know. 74% of 2016 Leave voters agreed with the statement, whilst 68% of Remain voters disagreed.

44% is a relative majority — a plurality — but is not “more than half of the public”. 54% is the agreed proportion once people selecting ‘Don’t Know’ are excluded.

Practice varies, but what needs to be explicitly clear to the reader is whether non-substantive responses are included or excluded. Claiming “a majority” support something when only a plurality does so is false.

Acquiescence Bias

Do you agree or disagree: we should avoid acquiescence bias? (Video: Pew Research Center)

There is a well-known problem in survey research called acquiescence bias. This is when respondents say ‘yes’ or ‘agree’ to questions, in order to be agreeable or to just get through the survey. This tendency can cause major problems in interpretation, particularly when majorities agree with conflicting statements.

Another agree-disagree statement question given in the survey was:

Brexit should be halted if problems over the Northern Ireland border threaten to split the Union

For this statement, 39% agreed, 37% disagreed and 25% said they did not know. This question indicates the public suffers from greater ambivalence that the question about delivering the EU exit “by any means” might imply, albeit one asking about a hypothetical scenario.

In conclusion

Reporting on polls, particularly on aspects of public opinion that are hard to measure, is challenging. Additionally, question design is a difficult art.

It would be helpful to the public if uncertainty and limitations were effectively conveyed.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store