COVID-19: Roche Antibody Test – 14th May

May 14, 2020

Susannah Fleming, Carl Heneghan

According to Roche, their new COVID-19 antibody test has “a specificity greater than 99.8% and a sensitivity of 100%.”

What do sensitivity and specificity tell us?

Sensitivity tells us the proportion of people who test positive, out of the population who should have tested positive. In this case, a sensitivity of 100% tells us that everybody with a history of COVID-19 infection had a positive antibody test. (Tests with sensitivity or specificity that are truly 100% are very rare. I suspect that in this case, the sensitivity is very high, but maybe not actually 100%.)

Specificity tells us the proportion of people who test negative, out of the population who should have tested negative. In this case, a specificity of 99.8% tells us that 2 in 1000 people (0.2%) tested positive with the antibody test, even though they should have tested negative.

Surely this is all fine? 99.8% and 100% are both big numbers. But what does it mean for an individual who receives an antibody test result?

Well, sensitivity and specificity can’t really help us. They tell us about what happens if we already know what the right answer should have been. But if you’re having a test, the whole point is that you don’t know the right answer. You get a test result, and you want to know how likely it is to be correct.

Interpreting an individual test result

For this, we need two different statistics: the positive predictive value, and the negative predictive value. These tell us the probability that the result is right, given that we have a positive or a negative test. Before we can calculate these, we need another piece of information, the prevalence. This tells us how common a condition is in the population.

We have some estimates of the prevalence of COVID-19, although we still aren’t completely sure of this. For the moment, let’s assume that the prevalence is 5%.

So if we have 10,000 people in our population. With a prevalence of 5%, that means, 500 should have antibodies, and 9,500 shouldn’t.

Being generous, and assuming Roche’s claim of 100% sensitivity, we know that all 500 of the people who should have antibodies will have positive tests. However, 0.2% of the people without antibodies (19 out of the 9,500) will also have a positive test.

Assuming the 5% prevalence, of those people who have a positive test, 96.3% (500/519) will have a “true positive”. This is the positive predictive value.

Because there are no false negatives, the negative predictive value is 100%

This isn’t as good as 99.8% and 100%, but it’s still pretty good. We can be pretty confident in both positive and negative tests. Most important in this case are positive tests since they might be used to decide if someone is safe to return to work or school, or to come out of shielding. However, around 4% (1 in 25) of positive tests may be false positives. It is important that people receiving the tests are aware of this.

How does this change for a different prevalence? If the prevalence is 10%, rather than 5%, then the situation is somewhat better. Because a higher proportion of the population is now expected to be positive, the number of true positives increases, while the number of false positives goes down a bit.

Our positive predictive value goes up to 98.2%, while our negative predictive value is still 100%

Even if we don’t trust the 100% sensitivity value, it’s likely to be quite high. Because there are so many more true negatives than the occasional false negative, even if we drop the sensitivity to 99%, there is no real effect on the negative predictive value – at worst it drops to 99.9%.

So how can an individual interpret their result from this antibody test? If they have a negative antibody test, they can be almost certain that they don’t have antibodies to COVID-19. If they have a positive test, it’s very likely that they do have antibodies, although there is a 2-4% possibility of a false positive.

What we still don’t know is whether a positive antibody test is associated with protection from future COVID-19 infection. And because we haven’t seen the study methods yet we can’t tell you whether these set of results are reliable or not.

About the authors 

Susannah   Fleming is a Senior Quantitative Researcher in the Nuffield Department of Primary   Care Health Sciences. Bio here

Carl Heneghan is Professor of Evidence-Based Medicine, Director of the Centre for Evidence-Based Medicine and Director of Studies for the Evidence-Based Health Care Programme. (Full bio and disclosure statement here)

Disclaimer:  the article has not been peer-reviewed and the sources cited should be checked. The views expressed in this commentary represent the views of the authors and not necessarily those of the host institution, the NHS, the NIHR, or the Department of Health. The views are not a substitute for professional medical advice.