Revisions and Stability

It is difficult to estimate how many new infections there are.

Robert Peston (ITV) wrote an article with the headline:

Covid: Did Boris Johnson order England’s coronavirus lockdown based on false data?

The answer is: no. In October, the COVID-19 infection survey showed rising positive test results. This was one of many sources which suggested the same pattern.

It is difficult to estimate new infections each day. This is different to how many people have an infection, in total.

The University of Oxford model of new infections appears unstable. This does not mean the policy rests on “false data”.

Rising infections in England

During October, the SARS-CoV-2 virus was spreading in England. More people were getting infections.

Several sources suggested infections were rising.

The Office for National Statistics runs an infection survey in private households. The goal is estimate the proportion of people who would test positive in the community.

The sample is a random selection of private households. Each person aged two or over can take part in the study. Participants take their own nose and throat swabs. Laboratories analyses these samples with polymerase chain reaction tests. All tests are imperfect. There are risks of false negative results — due to the timing and quality of swab samples.

This survey does not measure infections within hospitals, care homes, or other institutions.

The estimated proportion of people testing positive was over 1% by the end of October. (Image: ONS)

Between 25th to 31st October, 1.1% — 1.2% of people living in English private households would test positive. This proportion was under 0.1% at the start of September.

John Burn-Murdoch (Financial Times) highlights other sources showed similar rises in total infections.

Across sources, the estimated number of people with the virus was rising in October. (Image: Twitter/John Burn-Murdoch)

Also, Public Health England shows the positivity ratio was increasing in October. That means a higher share of people tested came back with positive results.

(Image: Public Health England)

Lagging indicators, such as hospital admissions, were also rising.

The number of COVID-19 hospital admissions in England counts people who:

  • Tested positive for SARS-CoV-2 in the 14 days prior to their admission;
  • Received a positive result after being in hospital.

On 30th October, the 7-day average for admissions was around 1,250 per day. Two weeks earlier, that average was about 770 per day.

COVID-19 admissions rose in England. (Image: PHE COVID-19 Dashboard)

Unstable models

The issue that Robert Peston highlights is with the daily model of new infections. The daily incidence model underwent major revisions:

Guess my surprise — indeed shock would better describe it — when I saw in the latest ONS survey, dated December 4, that the national statistician has downgraded its estimate of coronavirus in England on October 17 to just 4.89 people per 10,000.

And it now says the incidence of coronavirus in England barely increased until after the start of lockdown — and even during lockdown it says the prevalence never got above 6.62 per 10,000 (on November 12).

The official reported estimates have not changed.

The daily model of new infections provides a smoothed trend for the past six weeks. We can animate the daily incidence model for each publication date:

The graph shows the estimated number of daily new infections per 10,000 people. (Image: R Pubs)

These outputs appear to be unstable. There are large revisions from week to week. In the ONS report on 11th December 2020, there was no incidence estimate:

When enrolled on the survey, participants are swabbed weekly for five weeks and then move to monthly swabbing. Until recently, the majority of participants have been swabbed weekly providing us with regular and timely updates on the number of new infections and the “time at risk”. However, the proportion swabbed monthly has now increased. This means our estimation of incidence needs to be adapted to reflect the change in data capture to ensure the change to monthly swabbing has been fully reflected in our estimates. We are therefore taking further time to ensure we provide an accurate picture of incidence for the latest week and ensure our estimates continue to be of high quality.

It is difficult to estimate the number of new infections. More participants are taking monthly — rather than weekly — swabs. That change may have induced large revisions in the daily estimates. This was not the intention of the University of Oxford researchers.

Headline estimates concern the proportion of people who would test positive. The daily model of that measure does not undergo large revisions:

There are small changes from week to week. (Image: R Pubs)
Robert Cuffe explains it all. (Video: Twitter/Robert Cuffe)

The Office for National Statistics undertakes a large infection study. Other data sources also showed rising infections in October. Lagging indicators like admissions were increasing too. The daily model of new infections underwent major revisions — appearing unstable.

The R code for the animated graphs is available on R Pubs and GitHub.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store