Pluralities and Suspending Parliament
An article in The Telegraph claims that: “54 per cent of British adults think Parliament should be prorogued to prevent MPs stopping a no-deal Brexit.”
This article looks at the survey behind that claim.
Question Wording
ComRes conducted a survey of 2,011 GB adults via an internet panel, between 9th and 11th August 2019. This survey was sponsored by The Telegraph.
The article’s headline was:
Boris Johnson has public’s support to shut down Parliament to get Brexit over line, exclusive poll suggests
The polling question is worded somewhat differently:
Do you agree or disagree with the following statements?
Boris needs to deliver Brexit by any means, including suspending parliament if necessary, in order to prevent MPs from stopping it
This question wording is more elaborate than a simpler statement about support for suspending or ‘proroguing’ Parliament.
This statement about delivering the policy of exiting the EU — and the lengths (“by any means”) that the Prime Minister “needs to deliver” that policy. “Suspending parliament if necessary” (that is, in extremis) is not the same thing as thinking that prorogation “should” be done.
Exclusion of Don’t Knows
In The Telegraph article, and in the poll’s press release, it is asserted that ‘more than half of the public’ or ‘a majority of British adults’ support this suspension.
The survey results showed 44% of the sample agreed with the statement, 37% disagreed and 19% did not know. 74% of 2016 Leave voters agreed with the statement, whilst 68% of Remain voters disagreed.
44% is a relative majority — a plurality — but is not “more than half of the public”. 54% is the agreed proportion once people selecting ‘Don’t Know’ are excluded.
Practice varies, but what needs to be explicitly clear to the reader is whether non-substantive responses are included or excluded. Claiming “a majority” support something when only a plurality does so is false.
Acquiescence Bias
We also have to cautious when reading results of agree-disagree questions.
There is a well-known problem in survey research called acquiescence bias. This is when respondents say ‘yes’ or ‘agree’ to questions, in order to be agreeable or to just get through the survey. This tendency can cause major problems in interpretation, particularly when majorities agree with conflicting statements.
Another agree-disagree statement question given in the survey was:
Brexit should be halted if problems over the Northern Ireland border threaten to split the Union
For this statement, 39% agreed, 37% disagreed and 25% said they did not know. This question indicates the public suffers from greater ambivalence that the question about delivering the EU exit “by any means” might imply, albeit one asking about a hypothetical scenario.
In conclusion
Order effects were also suggested, but the statement order was randomised by ComRes.
Reporting on polls, particularly on aspects of public opinion that are hard to measure, is challenging. Additionally, question design is a difficult art.
It would be helpful to the public if uncertainty and limitations were effectively conveyed.
