MRP and Conservative Remainers

An article in The Observer showed constituency estimates for people who voted Remain in the 2016 EU referendum and then voted Conservative in the 2017 General Election.

A previous article in The Guardian considered Labour Leave voters in the same fashion.

Both articles provide little detail about the study’s methodology, called multi-level regression with post-stratification.

Here, I will give a short primer on MRP (or more fondly, Mr. P).

In short

• MRP outperforms sub-samples: This analysis provides more accurate and robust estimates of public opinion than simply looking at constituency (or state) sub-samples.
• Model the individual: MRP models the probability that an individual holds an opinion, based on their demography and geography.
• Estimate constituencies: Weighted averages of these individual probabilities are then calculated, giving an estimate of opinion in each constituency.

The Steps of MRP

Even with a survey of 150,000 people, that is only around 230 respondents per parliamentary constituency.

A full MRP primer by Kastellac, Lax and Phillips states:

MRP strongly outperforms disaggregation (i.e. produces estimates that are more accurate and robust) when working with small and medium-sized samples.

MRP does slightly better than in large samples, particularly when it comes to estimating opinion in small states.

The MRP procedure is:

Step 1: Gather a national opinion poll, recoding if necessary

The poll should include demographic information of the respondents, as well as an indicator of their geography (their constituency). That demographic information should only include what is available in the national census.

In the study, the YouGov panel included about 150,000 respondents.

Step 2: Create a dataset for constituency-level predictors

This datset is for constituencies. For models of EU referendum behaviour, we may wish to include UKIP vote shares in the 2015 General Election, utilise the Hanretty constituency estimates of Leave shares in the referendum (achieved through areal interpolation), or the real results by local authority area.

Step 3: Collect census data

To enable the calculation, we need to know how many people there are in each constituency with all the demographic characteristics.

As an example, if our model of Conservatives voting Leave was based on education level and age group, then we would need to know how many UK citizens without a university degree and aged 65 or over lived in each constituency.

Step 4: Build a demographic and geographic model of individuals

This study was interested to know, for each political party and 2017 non-voters, an estimate of 2016 EU referendum behaviour (Remain, Leave, or did not vote).

A respondent’s recalled vote is treated as a function of their demographics and constituency.

A regression analysis is used to find the best-fitting model. Previous regression analysis of EU referendum ward data suggests the most important factors were, in order: education level, age group and ethnicity.

From this model, probabilities that a 2017 Conservative voter backed Remain, Leave or did not participate in the 2016 EU referendum are estimated.

This is the regression part of MRP. This is multi-level because the survey response probability can vary beyond the demographic model in each constituency.

Step 5: Calculate weighted averages in each constituency

Using our census data (step 3) and our respondent model (step 4), we can now have:

• Numbers of each demographic type in every constituency;
• Modelled probabilities for survey response based on demography and constituency.

Through a weighted average, we can now calculate the estimated shares of EU referendum vote by 2017 Conservative electors. This is known as post-stratification.

Finally, we can give estimated levels of each type of voter (e.g. 2016 Remain and 2017 Conservative voters), in every constituency.

This methodology can be followed with vote intention (as YouGov demonstrated in the 2017 General Election), across different sub-national areas, and with other kinds of surveyed opinion.

Who did this study?

The study was conducted by Ian Warren of Election Data and Dr Kevin Cunningham, alongside Chris Curtis and Marcus Roberts of YouGov.

Additional assistance was provided by Ben Lauderdale, and the study was sponsored by Best for Britain.

At the time of writing, the full study with its methodology has yet to be published.