In a report by Onward, a conservative campaign group, it was claimed:
Voters (at a ratio of nearly 2-to-1) wanting a society that “focuses on giving people more security” (65%), rather than one that “focuses on giving people more freedom” (35%).
This article looks at the polling behind that claim.
Onward… to the data tables
Hanbury Strategy conducted their poll of 4,984 GB adults, between 21st and 28th June 2019. This meta-data is taken from the data tables, which differs what is written in the Onward report and on the Hanbury website.
Respondents were found using mobile app adverts. This survey was sponsored by Onward.
The question asked was:
Below are several pairs of statements. For each set of options, could you say where you own view lies on a scale. The closer you move the slider towards the more you agree with it.
Option A: I would rather live in a society that focuses on giving people more freedom
Option B: I would rather live in a society that focuses on giving people more security
People answering the survey used a slider to give responses between 0 (favouring Option A) and 100 (favouring Option B).
The full tables reveal that 65% figure includes people giving responses from 50 to 100. 50 is the middle option: its inclusion is inappropriate.
A different summary of that question is given in the main report. Indicatively by the graph, approximately 43% gave answers 61–100 favouring ‘more security’, and around 22% gave answers 0–40 favouring ‘more freedom’.
Samples and Scales
Using a different scale, with a different number of responses, may yield different answers. Krosnick and Presser’s survey design handbook states there is no standard practice for scale lengths: lots of different formats are used.
A psychological study by Preston and Colman published in 2000 found that test-retest reliability tends to decrease for scales with more than 10 responses.
There is an analytical challenge posed by such scales, given 49 and 51 are very similar responses between two options in a 101-point scale.
Also, a question was raised to me on Twitter regarding the low number of responses from those 75 or older. This is plausibly due to the sampling method: gathering responses from people using mobile apps. The large effect of the weighting (186 unweighted respondents represent 530 people in the weighted sample) affects the standard error of these estimates.
A sea change?
The results — where respondents favour ‘security’ over ‘freedom’ — are presented as “a sea change” in British politics. However, these are seemingly new questions, asked to respondents with a relatively new sampling method.
It is not a basis to argue that there has been a ‘sea change’. Instead, we should consider whether we have misjudged the tide.
As part of NatCen’s British Social Attitudes survey — an annual random probability sample — there has been long period of liberalisation of attitudes towards personal and public morality.
Typically, research into British public opinion has focused on two axes: economic liberalism (left-right) and social liberalism (libertarian-authoritarian). Broadly speaking, these axes are the extent that the government regulates the economy and regulates social mores.
The Hanbury Strategy poll and subsequent analysis may be somewhat relabelling these terms. However, some adults may not view ‘freedom’ and ‘security’ in conflict, leading to questions of validity.
New scales are difficult to construct, and may require more testing. Additionally, research is needed on possible effects of river sampling via mobile app ads.