Assertions of Perfection

Neither Abbott nor Roche antibody tests have “100% accuracy”.

Media organisations reported on supposed ‘100% accurate’ tests. The ITV headline was:

‘100% accurate’ antibody test could be coronavirus breakthrough

Neither Abbott nor Roche antibody tests have the claimed accuracy.

What is accuracy?

Manufacturers develop tests for the presence of antibodies. Antibodies are part of the body’s immune response to infection. In most cases, small numbers of antibodies remain in the body as memory cells.

Antibody tests (or serological tests) seek to identify those specific antibodies.

‘Accuracy’ can be a confusing term. Imagine you are testing for a virus. There are two main measures:

  • Sensitivity: the proportion of infected people with a correct positive result. This is the true positive rate.
  • Specificity: the proportion of healthy people with a correct negative test result. This is the true negative rate.

In formal terms: accuracy is the number of true positive plus true negative results, of all tests.

A ‘100% accurate’ test must give correct results to everyone tested. That would be a perfect test. Reality is often disappointing.

Roche and Abbott

Public Health England evaluated the Roche antibody test. That test seeks the presence of antibodies fighting the SARS-CoV-2 virus. The test concerns whether you have had the SARS-CoV-2 virus in the past.

Among negative samples, all did not react to the Roche test. The true negative rate has a plausible range of 99% to 100%.

The PHE analysis used 93 samples taken from recovering patients.

The true positive rate estimate was 84%. The plausible range runs from 75% to 91%.

In the analysis, the Roche test missed 15 samples who had the antibodies.

In general, the test gets better as time goes on. (Image: Public Health England)

For the Abbott test, 395 negative samples had no reaction. The headline true negative rate was 100%. The confidence interval runs from 99% to 100%. Yet, two of the 354 confounding samples gave positive results.

The overall true positive rate was between 86% and 97%. The central estimate was 93%.

There are general improvements over time. (Image: Public Health England)

There are limitations to the PHE analyses of these tests. As Prof Bird (MRC Biostatistics Unit, Cambridge) identifies, the two analyses differed.

There were more confounding samples for the Abbott test than the Roche test. There was no analysis by the patient’s age or sex.

Claiming perfection

It should be crystalline that neither test is ‘100% accurate’. Doctors and nurses can run take blood for antibody tests too early. There can be human errors in medical procedures.

The PHE analyses do not support saying these tests are ‘100% accurate’.

Pippa Allen-Kinross (Full Fact) has also written about antibody test accuracy.

This blog looks at the use of statistics in Britain and beyond. It is written by RSS Statistical Ambassador and Chartered Statistician @anthonybmasters.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store