Self-selection bias and social research

Some surveys use self-selection, meaning we cannot generalise.

A self-selecting survey allows respondents to put themselves into the sample. Another name for this kind of survey is an open access survey. This is common on social media, where users can generate their own ‘polls’. Websites can carry clickable surveys on pages.

In the past, newspapers ran text-in or phone-in surveys. Sir Robert Worcester (Ipsos MORI) labelled such surveys ‘voodoo polls’. Self-selecting surveys are cheap and convenient to run, but suffer from severe biases. We cannot infer from the self-selecting sample to the intended study population.

Self-selection bias and other problems

A self-selecting sample only contains those animated enough to vote. In standard surveys, potential respondents often do not know the topic in advance. People who are aware of open access surveys and choose to vote differ from those who do not.

That act of self-selection causes an unknown error. Researchers name that problem voluntary response bias, or self-selection bias. Such samples can contain too many engaged people or those wielding strong opinions.

British Future and HOPE Not Hate published a joint report in 2018. That report…