Vaccines, overconfidence and 98% cuts

An open letter makes a strong claim about vaccine effectiveness.

How much do vaccines cut deaths from COVID-19? An open letter from scientists, published in the Mail on Sunday, asserts:

We can be very confident that they will reduce Covid deaths by around 98 per cent and serious illness by 80–85 per cent.

This figure does not come from a vaccine effectiveness study. This post looks at those two claims.

How well do vaccines protect against death?

Vaccines can limit deaths in three ways:

  • Infection prevention: it stops people becoming infected with the virus.
  • Reducing severity: among people who still get infected, that infection is less severe — with fewer deaths.
  • Dampening transmission: of those infected, it can lower the likelihood of passing on the virus. That reduces further cases and later deaths.

The UK daily COVID-19 deaths measure reached a peak of 1,360 on 19th January. In the United Kingdom (without Wales), that counts deaths within 28 days of a positive test. Public Health Wales uses a different definition of confirmed deaths. That agency counts deaths with clinical suspicion COVID-19 contributed to the death. The centred seven-day average was 1,285.

UK daily surveillance COVID-19 deaths, by date of death, from 1st September 2020. (Image: GOV.UK)

By 13th April, that deaths measure fell to 20. The centred seven-day average was about 21. The reduction in rolling average deaths was around 98%. The letter appears to attribute the entire fall in recorded deaths to vaccines.

That ignores the impact of public restrictions. In England, there have been three national stay-at-home orders. Those lock-downs started on: 23rd March 2020, 5th November 2020, and 6th January 2021. Other restrictions, including local lock-downs and tiers, have also permeated.

Lock-downs and vaccines are a tag-team: cutting cases and making infections less severe.

In their analysis, Public Health England estimate from their effectiveness studies:

Vaccine effectiveness against mortality was based on PHE estimates of effectiveness of vaccination against symptomatic infection (58%) and of death given infection (54%) which combined gives 81% protection against death[.]

Vaccinations appear to lower the peak of deaths. (Image: Public Health England)

The mortality impact analysis does not account for dampened transmission.

What about the “serious illness” claim?

If we take “serious illness” to mean hospitalisations, the second range is about right:

Combined with the reduced risk of becoming a case (Section 3) this is consistent with vaccine effectiveness against hospitalisation of around 80%.

A prospective cohort study in Scotland estimated vaccine effectiveness against hospitalisations. Different periods after vaccination came to different numbers. After vaccination, there was an adjusted fall in hospitalisations:

  • 21 to 27 days after the first dose: 76% (70% to 81%)
  • 28 to 34 days: 89% (83% to 92%)
  • 35 to 41 days: 78% (68% to 85%)

It is very likely both vaccines and restrictions played a role in cutting deaths. In England, vaccine effectiveness against COVID-19 deaths is around 80%. That figure is for one dose: protection is likely to be higher after the full course.

Isolating the effect of one intervention is challenging. We should stress uncertainty, and avoid overconfidence.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store