Power Line

The Importance of Accurate Testing

The D.C. neurosurgeon whom I quoted here and whose mortality chart I reproduced here writes to comment on the need for, and difficulty of obtaining, accurate COVID-19 testing. Please note, especially, his conclusion:

The accuracy of testing for COVID-19 needs to be discussed, as all the numbers we are following (and the massive reaction to it) hinge on the tests’ accuracy. As of now, global testing for COVID is based on a tribal group of genetic tests, that differ by country, region, and laboratory. The CDC test, for example, tests a different set of genes, than say the German WHO test. These tests were rapidly developed under a lot of pressure, and potentially have error (as we already saw with the CDC test).

For the most part, the validation and specifications for these tests have not been publicly released, and probably have variation across methods and even laboratories. As a clinician, we look at the false positive and false negative rates of a diagnostic test. However, even very accurate tests can lead to a very high false positive rate, if the condition prevalence is low. How? The answer is through conditional (Bayesian) probability. Imagine if you have a population of 100 people, of which 1 truly has Disease X. Your test has a 5% false positive rate, and a 5% false negative rate. If you test all 100, ~6 will test positive (the one true positive, and the 5 false positive), but only 1 will be truly positive. This means, even with a 95% accurate test, there is an 87% chance that a positive test represents a false positive!

We know very little about the false positive and false negative rates of these varied tests. Similar PCR (gene) tests for common respiratory viruses have a 1-2% false positive/false negative rate (https://www.biomerieux-diagnostics.com/filmarrayr-respiratory-panel). For COVID, since the numbers being reported (e.g. on Worldometer) are positives, it is very possible that a large fraction of this number could represent false positives, in relatively low prevalence situations. It depends on adequate pre-test screening and limiting the test to high-risk populations (where have we heard this before?!). As we test more, we will find more positives — but only a fraction can be expected to be true, and the true positive rate will decrease as testing expands. As Dr.Birx (White House COVID Coordinator) said on Tuesday, “Quality testing,” she said, is “paramount.” “It doesn’t help to put out a test where 50 percent are false positives.” This is a very crucial point — with significant political implications. The CDC’s methodological and quality concerns are being mistaken for a “botching,” rather than a quality concern and a very basic epidemiological principle.

Leave a Reply