Some notes on excess mortality

As I prepared for an interview, I wrote notes on excess deaths.

Excess deaths are deaths from all causes above a baseline. That baseline often represents an expected number of deaths.

Different institutions use different baselines:

Excess death calculations differ through choices of periods and baselines.

Where can I find data on excess mortality?

There many institutions collating COVID-19 surveillance deaths. There is no single source for frequent all-cause mortality.

The World Bank calculates annual crude death rate per 1,000 people for countries. The United Nations Statistics Divisions publishes annual death statistics. The latest year in these datasets is 2019.

There are two teams of researchers collating weekly and monthly mortality figures.

News organisations, such as The Economist and The Financial Times, use these two sources and collate other mortality data.

Why do we need to look at excess mortality?

For public health agencies, COVID-19 surveillance deaths have specific definitions. Those definitions differ between countries, which makes for challenging comparisons. Definitions can also differ within countries too: with the USA and UK publishing aggregate statistics.

Timeliness matters more for surveillance numbers than completeness. It is useful to look at deaths from all causes, to circumvent labelling problems.

A major limitation is mortality statistics are not available for all countries. The UN Statistics Division estimates some countries have under half coverage of death registrations. Some places do not have sufficient death reporting systems to give accurate counts.

Civil registration is not complete across the world. (Image: UN Statistics Division)

What do we need to consider about excess deaths?

Excess deaths are not a way of directly measuring COVID-19 deaths.

The pandemic can increase deaths from other causes, such as pressuring healthcare systems. The pandemic can decrease deaths from other causes. Lower traffic can mean fewer fatalities. Interventions seeking to limit the spread of SARS-CoV-2 can also halt the spread of influenza.

Excess mortality — deaths above a baseline — is a useful lens to view the total impact of the pandemic.

What about the IMHE analysis?

On 6th May, the Institute for Health Metrics and Evaluation (IHME) published an “estimation of total mortality due to COVID-19”. It is very likely COVID-19 surveillance deaths under-count total deaths from the disease.

Through adjustments, the IHME analysis claimed there were 6.93m “total” COVID-19 deaths. The report says: “for now we assume that total COVID-19 deaths equal excess mortality”. There is much uncertainty surrounding this number.

Their analysis has four steps:

  1. Estimating excess mortality compared to observed deaths where that data is reported.
  2. Estimating the proportion of excess mortality that are COVID-19 deaths.
  3. Estimating the ratio of excess mortality to reported COVID-19 deaths.
  4. Generating predictions of excess mortality (assumed equal to total COVID-19 deaths) for all locations.

Each step represents statistical modelling. What is the uncertainty in their published estimates? Statistics is not alchemy: there is no circle to transmute what is uncertain into precision.

There are equivalent exchanges. (Image: Fullmetal Alchemist Brotherhood)

What about deaths in India and other countries?

Much press coverage focuses on the IHME estimate of excess mortality in India. Their data sources do not list an institution publishing mortality statistics for India.

Excess deaths are difficult to calculate for India. In mid-2020, the Civil Registration Service published death registration statistics for 2018. There is incompleteness too: an analysis in the Indian Journal of Medical Research suggested, in 2015, one in four deaths were not registered. For 2018, the CRS estimated death registrations were 86% complete.

Some city and regional agencies published annual figures up to 2020. Those places give a mixed picture: fewer deaths than the past average in Kerala, but more in Mumbai. That may be due in part to differential incompleteness.

There was both surging confirmed cases and positivity in India. That indicates some cases were being missed. The Financial Times finds cremations of Covid-19 victims are much higher in some cities than reported Covid-19 deaths.

FT analysis suggests cremations are higher than reported COVID-19 deaths. (Image: Financial Times)

Projecting deaths from cities to countries is challenging. Just as London had a different experience to Birmingham, Lucknow can have different mortality to Bhopal. India is a massive country: different parts will endure the pandemic differently. The World Mortality dataset has some local statistics for India.

Local data tends to be less complete than national figures. (Image: World Mortality Dataset)

The IHME excess mortality estimates differ from other known figures. For example, Karlinsky estimates there are around 605,000 excess deaths in the United States (from last year to latest weeks). The IHME report puts that figure at about 905,000.

Similarly, IHME gives an estimate of nearly 210,000 excess deaths in the United Kingdom. Comparing to the 2015–2019 average, we can calculate rounded figures for excess mortality:

  • Week 12 in 2020 to week 8 in 2021: 124,000
  • Only weeks in excess between week 1 in 2020 and week 15 in 2021: 127,000

For Japan, excess mortality between March 2020 and February 2021 (versus the 2015–2019 average) is about 54,000. Other analyses, including the Financial Times and The Economist, suggest Japan is experiencing deficit deaths. These superior analyses likely produce a baseline for 2020 by extrapolating trends. IHME gives an estimate for Japan of around 108,000.

Author’s calculations. Here, the study window is the first data period in excess (compared to the 2015–2019 average) to the latest such period. Sources: Our World in Data (World Mortality Dataset and government sources). Reported COVID-19 deaths are by publication date up to the latest day in the period. All death figures are rounded to the nearest thousand.

Over different periods, with a different baseline, analysts can calculate different excess death numbers:

Source: The World Mortality Dataset: Tracking excess mortality across countries during the COVID-19 pandemic (Karlinsky and Kobak, April 2021)

Calculating excess deaths during the pandemic is challenging. For some countries, that calculation may take years.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store