A Question of Wording
Questions on whether the British public should have a further referendum on exiting the European Union have a panoply of variants. How these questions are worded induces different responses, based on conceptual differences and mere choice in language.
This phenomenon was recently illustrated when the same company, surveying people in the same time period, asked two different questions on holding an EU referendum and received two different responses.
Consider the full wording of these questions:
- People’s Vote: If Britain looks set to leave the EU without a deal, would you support or oppose a public vote on whether Britain should leave without a deal or stay as a member of the EU?
- The Times: Do you think there should or should not be a new referendum held on whether Britain should leave the European Union or remain a member?
There are three differences between these questions:
- Condition: the former question sets the premise that Britain is about to leave the EU without a deal, whilst the latter does not;
- Means: the former question specifies that this referendum should have leaving without a deal on the ballot paper;
- A Public Vote: the former question uses the term ‘a public vote’, whilst the latter uses the term ‘referendum’.
As Prof Curtice demonstrated in UK in a Changing Europe’s January report on public opinion:
It seems that some Leave voters, at least, begin to warm to the idea of another ballot when it is implied that its purpose would be to enable voters (like themselves) to decide the fate of the Brexit deal that the government has negotiated. However, this support largely disappears when it is made clear it might result in a reversal of the decision to leave the EU.
Instead of repeating the same question, polling companies are often asking a large number of different questions with the same aim. Claims about a clamour of support for a new EU referendum should be treated cautiously.
The ‘Right to Vote’ campaign is part of the Best for Britain group, and seeks another referendum on exiting the EU. At the time of writing, they are running social media adverts claiming that a specific percentage of local voters want another referendum:
There is a curious statement about where this data comes from:
Source: Right to Vote — March 19
This is unhelpful. I asked their press officer what the data source actually was, and received the following reply (which is part of their press releases):
Analysis by FocalData from multiple research vendors, including Opinium, 2 January-4 March 2019. Sample: 14,046 adults from 632 constituencies across Britain. Findings based on preferences expressed.
Findings derived using multilevel regression and post-stratification (MRP) methodology, which is proven to produce very accurate constituency level findings. FocalData is a consumer analytics platform that uses machine learning to predict public opinion.
MRP does not, inherently, produce “very accurate constituency level findings”. It simply provides more accurate and robust estimates than looking at constituency sub-samples, which are often tiny.
Given the variety of question wordings, caution should be shown to specific claims of estimated support for a new referendum in each constituency.