European Social Survey and Education

The UK’s survey gives a high estimate for remaining in the EU.

A CNN headline claims:

Four years after Brexit, support for the EU surges in Britain

The headline is inaccurate, referring to the European Social Survey. For the UK, NatCen conducted the survey between August 2018 and February 2019.

The high Remain share results from, in part:

  • target population: a small number of respondents were 15 to 17.
  • response options: the interview was in person, with few unsure answers.
  • education structure: the sample has too many people with higher qualifications.

What is the European Social Survey?

The European Social Survey is a series of surveys run across many European countries. Each survey is a random probability sample, conducted every two years.

The UK has participated in each of the nine waves, funded by the Economic and Social Research Council.

In the ninth wave, NatCen conducted 2,204 face-to-face interviews of UK citizens aged 15 or over. These interviews were between 31st August 2018 and 22nd February 2019. The response rate was 41%.

The question for UK residents was:

If there were to be a new referendum tomorrow, would you vote for the UK to remain a member of the European Union or leave the European Union?

Image for post
Image for post

In the CNN graph, ‘Would not vote’ means every coded option that is not Remain or Leave. The survey data does not code the ‘don’t know’ answers.

Image for post
Image for post
I applied post-stratification weights. (Image: ESS Data)

If we exclude those other options, the estimated Remain share was 62%. This figure does not adjust for likelihood of voting in such a referendum.

Since this is a survey, there is a plausible range around each figure. It could be somewhat higher or lower.

62% seems rather high?

That is a high estimate. The eighth wave of this UK survey was for September 2016 to March 2017. The Remain estimate, on the same basis, was 53%.

The latest ESS wave was for August 2018 to February 2019. We can look at standard polling. After excluding undecided voters and non-voters, the Remain share ranged from 50% to 57%. The average was around 53%.

Image for post
Image for post
The What UK Thinks: EU website tracks hypothetical referendum polls. (Image: What UK Thinks: EU)

There are three partial reasons for this difference.

Target population

The ESS survey is of UK citizens aged 15 or over. In standard polling in Great Britain, it is of adults aged 18 or over. This difference has a small effect. Younger people prefer to remain in the European Union. With post-stratification weights, there were about 64 respondents aged 17 or under. Excluding responses not intending to cast a valid vote, the Remain share changes by less than 0.5 points.

Response options

Of around 2,200 respondents, only 66 said they did not know. That is 3% of the sample. By contrast, recent standard polls suggest higher rates:

Note these two polls use the terms ‘remain’ and ‘leave’, despite the UK already leaving the EU.

Image for post
Image for post
The full questionnaire is available to read on the ESS website. (Image: ESS)

Separation suggests interviewers may not state the respondent can refuse or be unsure. Populus research in 2016 suggested suppressing that option led to higher Remain estimates.

Education structure

As Matt Singh (Number Cruncher Politics) says, we can look at the working age population. For ESS respondents aged between 16 and 64:

  • About 50% have a degree, higher education diploma or certificate, or equal qualification.
Image for post
Image for post
Each country has a question about highest education level. (Image: ESS Data)

The Annual Population Survey estimates the proportion with Level 4 qualifications or higher:

  • January to December 2018: 39.2% (39.0–39.4%)
  • January to December 2019: 40.2% (40.0–40.4%)
Image for post
Image for post
The ONS estimates from NOMIS are very useful. (Image: NOMIS Query)

There appears to be too many people with higher qualifications in the sample. In the eighth wave, the UK survey did not weigh by education.

Educational attainment is associated with EU referendum choice. If the weighted sample has too many graduates, it will overestimate the Remain share.

The European Social Survey is a high-quality survey due to its design. Yet, education is becoming influential in political decisions. As a result, educational composition matters more in survey research.

Surveys provide estimates, subject to many sources of potential error.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store