Don’t Know? Don’t Know

How does presenting the Don’t Know option affect survey estimates?

In self-administered web surveys, how should researchers present the ‘Don’t Know’ option to promote accurate responses? This question sits at the heart of a methodology seminar held by Natcen and the European Social Survey.

Demands of a survey

There are two demands on a well-designed questionnaire:

1. An appropriate answer option should be available for all respondents;
2. People answering the survey should be motivated to give the most appropriate response.

These demands can come into conflict when trying to measure a lack of knowledge.

In interviewer-administered surveys, ‘Don’t Know’ is often not given as an explicit choice and must be volunteered spontaneously by the respondent. Sometimes, the interviewer then probes this initial Don’t Know response with further questions, trying to see if the respondent can reveal a more substantive answer.

This is much more difficult in self-administered surveys, with no interviewer to guide. Giving a Don’t Know option can lead to satisficing — a pleasant word for a troubling concept in survey research. People who satisfice are not giving true answers reflective of their real beliefs: they are clicking through the survey to get it done.