Cohort and Case-Control Studies

Observational analyses estimate vaccine effectiveness.

There are several studies of vaccine effectiveness in England and Scotland. There have also been analyses of the vaccine roll-out in Israel.

Despite having similar names, vaccine effectiveness is a different concept to efficacy.

Vaccine efficacy is about reductions in disease. A trial compares vaccinated volunteers to a control group. The trial places people into groups at random. Researchers estimate how well the vaccine does at reducing disease. Trials can under-represent or exclude some people. That affects translation from participants to population.

Vaccine effectiveness is about the programme. How do vaccine programmes affect health outcomes, outside of trial conditions? Those outcomes include people going to hospital and death.

Vaccine effectiveness is about how the programme helps people. (Image: WHO)

Observational analyses are different to randomised trials. There is no experiment to establish. The data comes from collection systems.

Observational analyses can suffer statistical problems. Hidden variables can influence both the ‘cause’ and ‘effect’, distorting the studied associations. In statistical terms, these variables are confounders. Statistical issues can induce systematic errors in estimates.

The three analyses in England and Scotland were all cohort studies. I will write about the Scottish study to explain.

In Scotland, the study looks at vaccinated people and COVID-19 hospitalisation. The vaccinated cohort got a vaccine between 8th December 2020 and 15th February 2021. It excludes people who had a prior positive test for SARS-CoV-2.

After vaccination, the study follows those people through time. Researchers check when (if at all) they had a hospital admission with COVID-19. The pre-print study compares likelihood of going to hospital, related to COVID-19. The analysis adjusts for age, sex, deprivation, and other factors.

We follow each cohort over time. (Image: Boston University)

After 28 to 34 days following one dose, effectiveness peaked. For both vaccines, COVID-19 hospitalisations reduced by around 84%. The interval was between 74% to 90%.

The ‘vaccine effect’ increased to its peak at 28–34 days. (Image: Twitter/John Burn-Murdoch)

In Israel, researchers conducted a case-control study. Using data from Claiht Health Services, they identified people who had a vaccination.

The study matched those vaccinated patients to ‘controls’ — people without the vaccine. This is demographic twinning: finding non-vaccinated people like those with the vaccine. The demographic matching was for age, sex, geography, and other factors.

For hospitalisation, effectiveness was around 74%. The interval was between 56% and 86%.

The graphs show the total incidence over time, for each group. (Image: NEJM)

When we hear effectiveness numbers in media reports, remember these figures are estimates. We should think of a plausible range of values, rather than focus on single estimates.

These studies show the vaccines give major protection against COVID-19.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store