Modeling The Pandemic: Efficacy

I’ve had a vision of writing a magisterial series to clarify everything about the mathematics of the pandemic, but it looks more like I will, hit-or-miss, do one issue at a time.

I’ve worked with mathematical models for chemical reactions. The math is identical to that of a pandemic – coupled differential equations describing the transformation of one set of things into other things. In the case of the pandemic that transformation is

susceptible people -> infected people -> immune or dead people

Sets of coupled differential equations have become popular for such things since we have gotten enormous computing power. Solving them isn’t easy to do by hand. Epidemiology (and chemical kinetics) grew up with simpler equations to model their processes. And those simpler equations can be good for some things!

I started out the pandemic trying to understand epidemiological modeling. It turns out that the models cover the range from simple equations to coupled differential equations to agent modeling. Last year, we heard more about models, but, like so much else in the pandemic, the reporters got enough wrong that news about the models ceased to be useful.

The value of models is in being able to look at various scenarios to evaluate paths forward. Reporters treated them as a horse race: which model got it closer to “right” today? With the definitions of “right” usually focused on a few numbers of interest to the reporters. We hear less about models now, although I’ve seen indications that the Biden administration is using them, which is good.

The big difference I see between chemical reaction models and epidemiological models is that the parameters of chemical reaction models are much better known or can be estimated reasonably well. The parameters of epidemiological models are known poorly and change, often in unpredictable ways. That has caused epidemiologists to do things differently than chemists do.

Today, let’s look at efficacy as defined by epidemiologists. The first thing to understand is that it is not the common dictionary definition. As for many other words appropriated by scientists, the definition resembles the common definition, which makes for confusion. Josh Marshall was struggling with this last week. I do not recommend his discussion.

Efficacy numbers are reported regularly and seem to play a part in the FDA’s deliberations for granting approved status. The CDC’s discussion of efficacy  is the clearest I’ve seen. Here’s the meat of it, but the whole thing is worth reading.

Vaccine efficacy/effectiveness is interpreted as the proportionate reduction in disease among the vaccinated group. So a VE of 90% indicates a 90% reduction in disease occurrence among the vaccinated group, or a 90% reduction from the number of cases you would expect if they have not been vaccinated.

I’ve seen other epidemiological writing that distinguishes between their definitions of efficacy and effectiveness, so there’s an additional potential confusion. More from WHO on efficacy. The illustration is from WHO.

What I usually want to know is something different: What is my probability of getting COVID even though I’ve been vaccinated? That requires a different set of data from a different set of observations. This definition of efficacy is helpful to me only in a qualitative way – some vaccines are better than others as indicated by their efficacy numbers.

This definition of efficacy is a marker of the results of large-scale trials. That’s useful and important, but limited. It’s also the number that can be obtained first. The number I want to know can be obtained only after a great many people have been vaccinated and exposed to the virus. But it is much more indicative of my risk and thus what I need to evaluate my activities.

As time goes on and more people are vaccinated, efficacy may change because the unvaccinated acquire immunity through natural infection. Fewer of them are infected, and the efficacy number goes down. However, an individual’s probability of being infected remains the same. This seems to be part of the difficulty in interpreting the numbers from highly-vaccinated Israel.

Takeaway: Efficacy in the epidemiological sense is useful for comparing vaccines; higher is better. But it should not be mistaken for a measure of breakthrough probability or the many other things we combine in the everyday meaning of efficacy.

Cross-posted to Nuclear Diner