Under the RAJAR
How many people listen to the radio? What do they listen to?
These are major questions for radio broadcasters and advertisers. Different programmes and stations are battling for listeners.
Journalists often write about radio listening statistics with total certainty. For example, a Music Week article claimed:
As for Chris Evans, he was… up! The former Radio 2 presenter added 3000 listeners as his Virgin Radio breakfast show edged up from 1,111,000 to 1,114,000. Marginal gains.
This article will explain where these listening figures come from.
Radio Audience Joint Audience Research (RAJAR) publishes radio listening figures for the UK. The BBC and Radiocentre own this research organisation. Radiocentre represent commercial radio stations.
Ipsos MORI conduct the fieldwork for RAJAR, finding adults and children (aged 10 or over) to survey in the UK. Whilst children are part of the survey, listening figures usually refer to people aged 15 or over.
Ipsos MORI select households to ensure coverage across the various radio services. Each interviewer gets up to 150 addresses across two small areas. Every fourth address highlighted for interviewers to try two calls. Other addresses are potential substitutes. There are stringent rules over the use of primary and alternative addresses.
Interviewers place diaries with one member in 15 different households, who has to be at 15 years old or over. Children and young adults can also asked alongside an older household member. Selected respondents may recruit others, increasing numbers within each sample. Survey researchers call this snowball sampling.
The 2019 Q3 survey covers 24th June to 15th September 2019. That survey has responses from 23,990 UK adults aged 15 or over.
Selected people answer a questionnaire, checking for eligibility and any potential conflicts. This questionnaire also records demographic details.
The interviewer uses a set of cards. These cards help people identify what radio stations they have listened to in the last year. Interviewers then give people labels for each station, for use in the diary.
People are then asked to complete a media consumption survey, and a radio listening diary. In the physical version, each day of the week covers two pages.
The diary breaks each day is up into 15-minute slots, filling in stations and what length of time people tuned in. Respondents must also complete how and where they listen to the radio, such as on a digital radio at home.
Interviewers encourage respondents to use the diary every day. People can also complete digital versions of the listening diary.
RSMB Ltd scan and check physical questionnaires and diaries. Concerns over data quality may mean recontacting interviewers and respondents. If there are doubts over the correctness of someone’s diary, RSMB Ltd excludes that data.
Some people write or type in the names of stations for which there is no label. Processors put such listening time to ‘BBC Local’ or ‘Local Commercial’, as appropriate. Unidentified stations go in the ‘Other listening’ bucket.
After data collection, RSMB Ltd applies weights to ensure population estimates are representative. RAJAR then publishes these estimates.
Design factors and uncertainty
Taking a survey has a cost: there is uncertainty surrounding each of these estimates.
The RAJAR estimates come from a complex survey. It is useful to publish that interval of uncertainty for simple random samples. A simple random sample means everyone in the population had an equal chance of being in the survey.
Yet, it is often more appropriate to account for the survey design’s complexity.
Since RAJAR oversees a complex survey, that interval is wider than for a simple random sample. Survey researchers call the ratio between the standard errors the design factor.
In this case, the design factor is 1.64. This means the uncertainty interval is 1.64 times wider than for a simple random sample.
Imagine a survey of 1,000 people. It is common to hear uncertainty involved in a simple random sample of this size: plus or minus three points. What this actually means is: for an estimate of 50%, the plausible range for the real value is 47% to 53%. The uncertainty for smaller proportions is lower.
Given the design factor of 1.64, the uncertainty surrounding an estimate of 50% runs from 45% to 55%. This design factor is for the national survey.
In June-September 2019, RAJAR estimated there were 1.11m Chris Evans Breakfast Show. This programme had a plausible range of between 0.95m and 1.27m people tuning in.
The best estimate was a quarterly increase of 3,000 listeners. Yet, the real number of listeners could have fallen, or increased by more than that.
Radio listening figures come from the combined work of RAJAR, Ipsos MORI, and RSMB Ltd. These figures are survey estimates, subject to many sources of potential error.
Journalists must take care when quoting these figures. People should avoid implying minor changes in listener estimates are real. For these small changes, natural differences between samples may be a better explanation.
Broadcasters and journalists should consult with RAJAR if there are doubtful claims.