Is a 60% risk reduction really no big deal?

Paul Alper writes:

Here’s something really important.

Notice how meaningless the numbers can be. Referring to a 60% risk reduction in flu due to the flu vaccine:

As for the magical “60?” Dr. Tom Jefferson didn’t mince words: “Sorry I have no idea where the 60% comes from – it’s either pure propaganda or bandied about by people who do not understand epidemiology. In both cases they should not be making policy as they do not know what they are talking about,” he said, insisting that I quote him.

Or, you could look here:

Researchers reported in the New England Journal of Medicine (August 14, 2014) that a high dose flu vaccine was more effective than the standard flu vaccine for seniors. The vaccine is called Fluzone High Dose vaccine. Of course, the media jumped on this report. In the Healthday article, the chief medical office for Sanofi-Pasteur—the Big Pharma company who funded the study—stated, “The study demonstrated a 24 percent reduction [emphasis added] in influenza illness among the participants who received the high-dose vaccine compared to those who received the standard dose.” . . .

1.4% of the seniors who received the high dose vaccine became ill with the flu and 1.9% of the seniors who received the standard flu vaccine developed the flu. (I hope you are not laughing as I did when I read that.) How in the world did they report a 24% lowered incidence of the flu with the use of the high-dose vaccine? Simply dividing 1.4% by 1.9% gives the relative risk reduction of 24%. However, this is a relative risk reduction—a useless number to use when deciding whether a therapy is good for any patient.

Dr. Brownstein, the source of the above quote, is an advocate of holistic medicine and supplements, so he says:

Folks, don’t be fooled here. This study was another failed flu vaccine study. The flu vaccine has never been shown to protect the elderly from getting the flu, dying from the flu, or developing complications from the flu. The elderly would be better served by eating a better diet, maintaining hydration and taking vitamin C.[!!]

I replied: I’m confused. Setting aside bias, sampling error, etc., a reduction from 1.9% to 1.4% is pretty good, no? It’s not a reduction all the way to 0, but why would this be called a failure?Also the diet, hydration, vitamin C thing seems irrelevant to the vaccine question, in that you could do all these things and also take the vaccine. What am I missing?

Alper wrote:

The actual 1.9% to 1.4% is far less impressive (and possibly not reported) than the 24% reduction touted in a press release. Relative risk is in general a misleading number. Absolute risk should always be stated as well. According to the links I listed, relative risk should never stand by itself without asking, “relative to what?”

Different example: suppose a disease is very rare, 2 in a million and a treatment reduces the incidence to 1 in a million. Huge relative risk reduction but hardly any effect on absolute risk. And don’t forget that vaccinations are not without “harms” so any benefits should also be compared to problems due to the treatment. And then there is the cost of the treatment which is often not reported.

I included that vitamin C quotation of Brownstein to indicate his “holistic” bias. Personally, I don’t trust Big Pharma at all, but the alternative medicine advocates are often so kooky that conventional medicine can look good in comparison. Besides, even great scientists (Pauling) lose all their marbles when it comes to the benefits of vitamin C.

I replied:

Sure, that I understand, but it also depends on seriousness. Suppose for example that 1.9% of seniors _died_ of the flu in a given year. That would be a huge number, a large proportion of total deaths, and a reduction to 1.4% would likewise be a big deal. Indeed, framing it as 1.9% vs 1.4% would be a bit of a minimization.

Alper:

We agree that researchers should always report relative and absolute risks as well as costs of treatment in addition to harms due to treatment. To often, the hyping of a treatment omits costs and side effects, emphasizing relative risk reduction which always sounds more impressive. This particular case refers to high dose vaccine (for people like me, an octogenarian) vs. ordinary dose (for you youngsters). I believe the percentages refer to infection rather than death. And for no good reason other than ignorance at the time, I selected the high dose. The price was the same, i.e., zero for those on medicare.

Certainly, with a large population the extra .5% would save a lot of people dying from a lethal infection but even then we need to know the cost of treatment. Indeed, as my very first link on this subject, a physician offers up:

What I long for—and I haven’t seen it yet—is for media coverage this season to start reporting on absolute differences related to the flu vaccine. I’d like to see how the “1-3% effectiveness of the vaccine” floats around in the public’s thought bubbles. How does that compare with something as simple as staying home and not infecting other people or washing your hands more frequently?

20 thoughts on “Is a 60% risk reduction really no big deal?

  1. I didn’t read the original paper but as a public health physician, I am neither shocked nor surprised by the finding. Surely what is given is a relative risk reduction (of 26% rather than 24%) and not a relative risk, but I think it is here only medical daily jargon among scientist who know what they are talking about. But the clinical effect, if not very large, is not negligible. Here, one must take account of the high prevalence of flue and the overall burden of the disease in the population. In the present context, lowering the prevalence from 1.9 to 1.4 can make a difference when judged by public health standard. These all depends, of course, on the reliability of this big pharma study.

  2. It should not really be an individual decision analysis but rather a population based one – my _guess_ is it will have a huge effect on health care costs (including waiting times), morbidity and mortality _if_ adopted widely.

    Perhaps not the best _advertising_ to use to affect individual consumer choices though!

    > compare with something as simple as staying home and not infecting other people or washing your hands more frequently?
    Evidence for an intervention that increase the percentage that would comply with that? (And a cost/benefit analysis perhaps comparing it the above.)

    Disclaimer: I use to work in vaccines.

  3. A cursory search suggests that there are about 50 million seniors living in the US. If you managed to vaccinate all of them, a change from 1.9% to 1.4% would mean a quarter of a million fewer cases of the flu.

    • And yet, I bet that if you did a survey asking people how much they would pay for a vaccine which reduces their risk to the flu to 1.9% and asked another group how much they would pay to reduce their risk to the flu to 1.4% you would get quasi-identical numbers.

    • Here’s a paper on the estimated cost-effectiveness of switching to high-dose version of the vaccine for the elderly in the United States: http://www.sciencedirect.com/science/article/pii/S0264410X14014790

      “Compared to IIV3 [low dose], HD [high dose] would avert 195,958 cases of influenza, 22,567 influenza-related hospitalizations, and 5423 influenza-related deaths among US seniors. HD generates 29,023 more Quality Adjusted Life Years (QALYs) and a net societal budget impact of $154 million”

      • Good find. In the absence of substantive criticisms of this study or alternative studies with different cost-benefit calculations, this seems like the definitive answer.

        There’s a $100M bill sitting on the sidewalk.

        At the very least, this would seem to justify a very large scale mass HD vaccination experiment.

      • This analysis seems to be essentially finding a correlation between percent of specimens that test positive for flu each year vs normalized deaths (the number of deaths in Nov-Apr that year minus number of deaths in May-Oct; start each “year” in May or Nov). They see when a higher percent of lab tests detect flu for a year, there are also more deaths.

        The CDC recommends against this:
        Dushoff et al (2005) write: “Following Thompson et al., we use the proportion of tested samples positive as a proxy for prevalence of each influenza subtype.”

        CDC writes: “The percent of specimens testing positive for influenza is not reported because public health laboratories often receive samples that have already tested positive for influenza at a clinical laboratory and therefore percent positive would not be a valid indicator of influenza activity.”
        http://www.cdc.gov/flu/weekly/#S1

  4. My email interchange with Andrew includes Dr. David Brownstein’s negative comments regarding the difference between the high dose vaccination vs. the ordinary dose vaccination. I noted that he is an advocate of alternative medicine (and I am not). He also is a critic of the MMR vaccine (and I am not):

    http://blog.drbrownstein.com/1029-2/

    http://blog.drbrownstein.com/toxic-vaccines-and-autism-a-cdc-coverup/

    “The CDC keeps putting out articles saying mercury-containing vaccines are safe. I think it is ludicrous—how can they be safe? It doesn’t make common sense nor does it make biochemical or physiological sense that injecting mercury into a human being is safe. Someday, the history of medicine will look on this practice of medicine as the dark ages of medicine. (Note: The MMR vaccine does not contain mercury). Now, there may be proof that the CDC not only knew about the link between the MMR vaccine and autism but they changed the data in a landmark 2004 study to hide the damning data. What did the heads of the CDC do? They altered the data and reported in 2004 (1) that there was no association between autism and the MMR vaccine.”

    We live in an age where cover-ups are alleged to be everywhere and statistics may or may not help to resolve the issues.

  5. For what it’s worth, a colleague of mine has estimated an average of 40K deaths/year attributable to influenza in the US 1979-2001. Given that many of these are seniors, a 24% relative risk reduction could indeed be a really big deal.

    From the abstract of “Mortality due to Influenza in the United States—An Annualized Regression Approach Using Multiple-Cause Mortality Data”, Dushoff et al. Am J Epidemiology doi: 10.1093/aje/kwj024

    The authors have extracted multiple-cause death data from public-use data files for the United States from 1979 to 2001. The current research reevaluates attribution of deaths to influenza, by use of an annualized regression approach: comparing measures of excess deaths with measures of influenza virus prevalence by subtype over entire influenza seasons and attributing deaths to influenza by a regression model. This approach is more conservative in its assumptions than is earlier work, which used weekly regression models, or models based on fitting baselines, but it produces results consistent with these other methods, supporting the conclusion that influenza is an important cause of seasonal excess deaths. The regression model attributes an annual average of 41,400 (95% confidence interval: 27,100, 55,700) deaths to influenza over the period 1979–2001.

  6. Clinical researchers, per se, are in no way qualified to determine the cost of treatment. They should avoid publishing stupid statements concerning it, and leave that to the economists. [Sic: publishing stupid statements.]

    They do not know the cost of vaccine manufacture at various scales, nor the cost to a medical facility of getting the vaccine to a patient, nor the patient’s cost of getting to the medical facility. The latter two points no doubt vary widely depending on factors such as whether the patient lives at home or in a facility. Some of those factors likely correlate with the incidence of the disease, and so the value of the treatment, as well.

    Then there is estimating the cost contracting the disease…

  7. Mr Alper displays his ignorance here by taking a shot at vitamin C – ascorbate. Pauling has been dead for several decades and his enemies are still attacking him! If you read Primal Panacea by Dr. Thomas Levy, you will find out that the science proves that ascorbate does much, much more than make seniors healthier. It kills (inactivates) viruses when its level reaches saturation in the blood. This makes it a potent competitor to the flu shot. And it doesn’t carry any risk factors like the flu shot. The CDC and WHO are just lying about this. If you want to see exactly what the first large vaccine (polio shot) really accomplished versus C, read Dr. Frederick Klenner’s presentation to a medical symposium on how he cured 60 cases of polio in 1947. The audience, who wanted to develop a vaccine, ignored him. How many people died and got paralyzed from polio since that shameful episode?

Leave a Reply

Your email address will not be published. Required fields are marked *