Sampling Error and Small Proportions

Sampling error is caused by observing a sample (some of the population) instead of the full population. It is inherent uncertainty, arising from running a survey — rather than a census. We are usually interested in how much the survey estimate could plausibly differ from the population.

There are three notable misunderstandings when people talk about surveys, such as political opinion polls: the sampling error of small proportions, the treatment of sub-samples, and total survey error. This article examines these three misunderstandings.

Margins of sampling error are not uniform

We are Flint commissioned a survey of 2,007 UK adults, conducted by Delineate via an internet panel. The main result was about 4% of respondents could spontaneously offer the name of a UK think tank. In the survey sample, the most recalled think tank was the Adam Smith Institute, with about 1% of respondents.

For some reason, the results were spuriously published to two decimal places. This suggests a degree of precision that survey estimates (of that size) cannot offer.

However, the accompanying notes stated:

All of the numbers included are within the margin…