Prompts and Political Polling

In survey research, how people answer questions can depend on how those questions are worded, what order the questions appear, and what response options are offered.

This article looks at polling performance in the recent European Elections and the different methods employed by UK polling companies.

Overestimation of the Brexit Party

Nigel Farage, leader of the Brexit Party, suggested there was an “establishment attempt to suppress the truth” — after YouGov had the Brexit Party in their secondary prompting.

This is a strange accusation, given most polling companies overestimated Brexit Party support in the recent European Elections.

YouGov overestimated the Brexit Party’s support in that election by five points.

How do different companies prompt?

In internet surveys, people self-administer their answers by clicking options. For vote intention questions, these options appear as radio buttons for people to select. In phone polls, the options will be read aloud to the respondent, to which they respond.

For Westminster vote intention, most polling companies provide a reasonably long list of options for people to choose: Conservatives, Labour, Liberal Democrats, UK Independence Party, Green, the Brexit Party, and Change UK. In addition, Welsh respondents are typically offered Plaid Cymru and Scottish people are able to choose the Scottish National Party.

Companies may offer a shorter initial list, taking two approaches to people wanting to vote for another party. Five examples are given, correct as of 1st June:

  • Secondary prompting [YouGov, BMG Research, Survation]: The first YouGov list contains Conservatives, Labour and the Liberal Democrats. If people select they wish to vote for another party, a second list appears. This secondary list currently contains UKIP, the Brexit Party, Greens, UKIP and the two national parties. For BMG Research, the primary list is the Conservatives, Labour, Liberal Democrats, and SNP/Plaid Cymru. Their secondary list is the Brexit Party, Change UK, Green and UKIP. Survation places the SNP — but not Plaid Cymru — in its primary prompt.
  • Spontaneous response [Ipsos MORI, Kantar]: The shorter list of parties contains Conservative, Labour, Liberal Democrats, national parties, Green and UKIP. People intending to vote for Change UK and the Brexit Party must spontaneously respond to that question, by saying or typing.

Polling companies may use different lists for different types of elections, and may change these approaches in future.

Why can prompting lead to overestimation?

If secondary prompting is undertaken, the question arises as to which parties should sit in the primary list. YouGov have been in a similar situation with another Eurosceptic party led by Nigel Farage.

Back in 2014, UKIP won the most votes in the European Elections. Even in YouGov’s secondary list, UKIP were receiving around 15% in opinion polls across that year.

Starting in 2015, UKIP were then elevated to the primary list. It should be noted that the party’s estimated vote intention share fell slightly after this change. Whilst YouGov’s UKIP vote intention share was reasonably accurate in 2015, the primary status persisted.

Following the calling of the 2017 General Election — with the Conservative government initiating the UK’s withdrawal from the European Union — the UKIP vote intention share immediately fell.

In the snap election, UKIP candidates only covered 378 constituencies (out of 650). People may have intended to vote for this party, but found themselves unable to do so. After votes were counted, there was some overestimation of UKIP’s vote share in pre-election polling.

The goals of survey research are accuracy and insight, not pandering to politicians. It is for social research companies to test methodological changes, and determine the most accurate means of measuring support for new (or reincarnated) parties.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store