Self-selecting surveys and sample sizes

The size of a self-selecting sample is often unimportant.

Richard Tice, the Reform party leader, asserted on Twitter:

FASCINATING: huge poll, totally at odds with YouGov ( Govt propaganda arm) that says 71% favour them. Is YouGov so discredited it’s better to believe the opposite of many of their polls?

The cited YouGov survey had a different subject (nightclubs) and wording:

Would you support or oppose people being required to show they have received both doses of the Covid-19 vaccine as a condition of entry into nightclubs?

To reiterate: self-selecting surveys are not reliable ways to measure public opinion.

A self-selecting survey means respondents choose whether to take part. They put themselves into the sample. There are many examples, including text-in ‘polls’ and clickable website surveys. Social media sites may allow their users to generate a survey, which others can vote in.

We can make no inferences from self-selecting surveys to the general population.

There are many statistical issues with these surveys, including self-selection bias. People who see a survey on social media and choose to vote are different to those who do not. These people are more likely to be animated by the question. That leads to a disproportionate number of extreme responses.

Also, there are few limits on participation. Accounts clicking on Gary Neville’s Twitter survey may not be adults nor in the United Kingdom. Social media users are not representative of the general population. There is no demographic information to ‘weigh’ responses either.

The irrelevance of a self-selecting sample size

Mr Tice suggests the number of accounts voting is somehow relevant (“huge poll”).

For random surveys, there is sampling error. That is the difference between the selected…