Skip to content
 

All cause and breast cancer specific mortality, by assignment to mammography or control

Paul Alper writes:

You might be interested in the robocall my wife received today from our Medicare Advantage organization (UCARE Minnesota). The robocall informed us that mammograms saved lives and was available free of charge as part of her health insurance. No mention of recent studies criticizing mammography regarding false positives, harms of biopsies, etc.

I bring this up to illustrate that statistics have failed to dent the mystique and eagerness of the mammography lobby’s incessant push to overtreat and overdiagnose. Below are two famous graphs from a 25-year Canadian study.

Wow. I’d like to see the link to the source of these graphs, along with the raw data. But assuming they’re correct . . . wow. I mean, sure, we can come up with all sorts of stories, and mammography has gotta be better now than it was 25 years ago. But still, no visible difference at all. . . . wow.

32 Comments

  1. D Kane says:

    > wow

    I am surprised that you are surprised by this. A standing finding since the famous RAND experiments is that almost all health care spending does not make us healthier. (There is no evidence, for example, that annual check-ups do anything useful.)

    Robin Hanson has been making this point for more than a decade:

    So I want to say loudly and clearly what has yet to be said loudly and clearly enough: In the aggregate, variations in medical spending usually show no statistically significant medical effect on health. (At least they do not in studies with enough good controls.) It has long been nearly a consensus among those who have reviewed the relevant studies that differences in aggregate medical spending show little relation to differences in health, compared to other factors like exercise or diet. I not only want to make this point clearly; I want to dare other health policy experts to either publicly agree or disagree with this claim and its apparent policy implications.

    This argument holds especially for things like mammograms that a) are diagnostic/preventative and b) have powerful groups that benefit from their use. How much money would GE lose if we recognized that mammogram machines don’t improve outcomes? How much would the incomes of doctors drop if they could not charge for mammograms?

    • Dzhaughn says:

      The Single-payer systems in the US (Medicare and the VA) pay for healthy mammography. Any Obamacare qualified plan has to pay for it. That is because “mammography-just-because” is driven by consumer demand. Doctors prescribe it because their patients (as a group) demand that they do so.

  2. Dale Lehman says:

    Having recently gone through the male version of this (PSA testing and prostate cancer), I wouldn’t jump to conclusions too quickly. Similar studies have shown that PSA testing may be counterproductive since the cancer is slow growing and you are likely to die from something else. This has led to a backlash and routine PSA testing is no longer automatically recommended. However, more recent data appears to be showing that the change in practice may have been a mistake – lack of testing and treatment may be associated with longer term increases in the development of prostate cancer (I did notice the upward tick in breast cancer survival at 25 years and wonder what the data looks like). More importantly, it is far from clear how to interpret the aggregate data when there are so many individual confounding influences (e.g., what is the state of health of the participants in the trial?).

    The truth is that both treatments are fraught with uncertainty and that the average effects may not be particularly useful at an individual level. The incentives for providers to over-diagnose and over-treat are clear. But it is also clear that many people have an incentive to advocate for stopping the routine testing. Both mammograms and PSA testing are open and active controversies and I would resist any simple declaratory answers. As has been discussed before on this blog, medical practice does not deal well with uncertainty – yet uncertainty is endemic to these issues.

    • Seb Nickel says:

      “But it is also clear that many people have an incentive to advocate for stopping the routine testing.”

      Who are you think of?

      • Dale Lehman says:

        Any of the myriad people opposed to expanding access to health care, interested in decreasing the cost of public (and to some extent private) insurance prices, free enterprise think tanks, etc.

    • Martha (Smith) says:

      “it is far from clear how to interpret the aggregate data when there are so many individual confounding influences (e.g., what is the state of health of the participants in the trial?)”

      One possible confounding influence that I have been wondering about is age. Were the women in the trial all the same age cohort?(I would guess not, but might be wrong.)

      As with many situations, one data set from a study looking at one aspect of a situation needs to be considered in combination with studies of other aspects of the same situation. Some examples of other types of studies relevant here:

      Studies of breast cancer detection at different ages.

      Studies of breast cancer death at different ages.

      Studies of how breast cancer is detected. (I recall hearing once that most cases of breast cancer are detected not by mammogram nor by intentional manual breast exam, but by the woman noticing a lump while showering.)

      • Dale Lehman says:

        Also, there is variation based on who is reading the mammogram. I don’t know what the evidence is for mammograms, but MRI results vary considerably depending on the experience and expertise of who is reading them. It is likely that these trials do not capture that source of variation.

        • Martha (Smith) says:

          Good point. The following quote from http://emedicine.medscape.com/article/1948247-technique supports your point:

          “Double Reading

          Double reading means that the mammogram is read by 2 radiologists either independently or interpreted together. As opposed to the United States, it is a standard practice in Europe. The aim is to increase the sensitivity and specificity of the examination. If the readers happen to differ in their interpretation of a mammogram, the issue can be approached in 3 different ways. Patient may be called back for follow-up studies if either of the radiologists identifies an abnormality, a strategy termed ”highest reader recall.” Alternatively, ”arbitration” is provided by a third reader who reviews the films and determines if the patient needs further diagnostic work-up or a ”consensus” is developed by a panel of radiologists that may or may not include the original readers.

          Many observational studies have shown benefit for double reading, but no randomized trials have looked at the practice. Data review from the Norwegian Breast Cancer Screening Program for over a million examinations that were performed by radiologists, 50% of whom were dedicated mammography readers, found that 23.6% of the cancers detected were diagnosed in patients who were recalled for discordant interpretation. [13]
          Computer-Aided Detection

          CAD or computer aided detection is a computer-based technology that helps the radiologist in identifying suspicious areas while reading a digitalized mammogram. It was approved by the FDA in 1998. No randomized trials have been performed to assess its effect on breast cancer mortality. A recent meta-analysis showed a small statistically insignificant increase in cancer detection rate, but this was associated with a higher recall rate and more false-positive readings. [14] “

  3. Clyde Schechter says:

    The Canadian study is just one of many studies. There have been several other randomized controlled trials of mammography and numerous observational studies involving comparison of breast cancer mortality rates in different parts of the same country when a national screening program is introduced at different times. There is also more indirect evidence: the decline in breast cancer mortality in the US during the final decade of the 20th century is too large to be explained just by improvements in treatment that were available then, unless you want to postulate that those treatments were much more effective in the real world than they were in clinical trials. (The opposite is the usual case.) It is also impossible to reconcile the observed patterns of incidence and mortality over time with complete ineffectiveness of mammograms unless one assumes that the relentless, steep, steady secular increase in breast cancer incidence prior to 1975 abruptly came to a halt in 1975 for no particular reason and never resumed. That is, of course, possible, and, to be fair, it is not without precedent among other cancers, but it is unusual and casts further doubt on the absence of mortality benefit from mammograms.

    Overall the evidence is really quite contradictory, and all of the trials and observational studies have serious limitations that threaten their validity. My own opinion, having delved pretty deeply into all of this, is that probably mammograms do save lives, but not as many as most mammography advocates would have you believe, and at a cost in overdiagnosis that few adequately acknowledge. I think they are substantially overused, but not useless.

    In any case, to have a strongly held opinion about this issue is foolish.

    • “the decline in breast cancer mortality in the US during the final decade of the 20th century is too large to be explained just by improvements in treatment that were available then, unless you want to postulate that those treatments were much more effective in the real world than they were in clinical trials”

      Or, that decades of improvement in health behavior such as reduced smoking, diet change, exposure to lead in paint, … whatever led to reduced cancer in the population. Population rates through time are affected by gazillions of things, the assumption that health care policy is the dominant effect seems suspect.

      • Clyde Schechter says:

        Except that there have been studies of the associations among smoking, and various aspects of diet and breast cancer and the findings are that these relationships are weak. Adding those things in does not come close to closing the gap.

        Smoking reduction has had massive impact on public health as a whole, and particularly on heart disease and lung cancer, but the impact on breast cancer is very modest. Smoking is the root of most, but not all, evil.

        • Undoubtedly you have more background here than I do. I just think that the uncertainty about causes is very large and as you say “In any case, to have a strongly held opinion about this issue is foolish.”

          Still, I think the prior on mammography needs to start out with 0 effect firmly in the high probability region given these large randomized trials.

    • Carlos Ungil says:

      “The lack of a decline in advanced cancers in the face of an increase in localized cancers suggests that public health initiatives aimed at prevention and early detection, while highly successful in terms of implementing a stage shift, have not contributed to the decline in mortality. The observed decline in mortality is based to a large extent on drugs introduced in the 1970s and 1980s. The protracted decline in case fatality was likely due to incremental adjustments in the drug doses and duration (alone and in combination) as well as a gradual expansion in the number of women who were candidates to receive adjuvant hormonal and/or chemotherapy.”

      http://www.sciencedirect.com/science/article/pii/S2213538315000065

    • Thanatos Savehn says:

      Colorectal cancer incidence and mortality has also seen a decline that can’t be explained by screening: http://www.nejm.org/doi/full/10.1056/NEJMp1600448 I wouldn’t dismiss the relatively sudden peaking and subsequent decline (~40%) in this deadly disease just because “no particular reason” can be given for it. Gastric cancer saw an even steeper and earlier decline https://academic.oup.com/epirev/article-abstract/8/1/1/462074 and nobody gets screened for stomach cancer. Something is clearly afoot.

  4. Dzhaughn says:

    This is a case of zero net benefit, not zero effect. The institution of mammography does reduce breast cancer mortality; it is overall beneficial in sub-populations where cancer is more likely. Unfortunately, it currently leads doctors and patients into other actions whose negative consequences outweigh this benefit.

    Appropriately for this blog, the crux of the problem is the difficulty of reasoning in the presence of uncertainty. Here, the interpretation of the mammogram is uncertain, there are many individually rare negative outcomes of various procedures. Good luck with what is often the best answer: the mammogram is not definitive but the only indicated action is doing another mammogram next year.

    • Dale Lehman says:

      The situation is worse with PSA testing – the indicated action is doing another prostate biopsy next year…. (I’ve seen people that have had over a dozen biopsies and I’ve also seen at least one person who said “if the choice is between death and another biopsy, I’ll choose death”)

      • Anoneuoid says:

        It’s seemingly an important gateway to the healthcare system though. I was told by someone (~70 y/o) they almost never went to the doctor until one time they broke an ankle. Once there they got convinced to get a prostate exam. The exam results indicated radiation therapy, so they got that. Then a growth formed on their colon (this was attributed to the radiation), so they also have to go in for regular colonoscopies. After awhile of this type of thing their blood pressure went up, so now they are also on blood pressure meds and need to check in intermittently for that… In a few years they’ve gone from spending almost nothing on healthcare to being a major client. This almost sounds like an “ideal” funneling process.

    • Clyde Schechter says:

      This is what I was referring to in my earlier post where I said mammograms are overused. Nearly all of the benefit of mammograms is likely concentrated in a fairly narrow age range and certain population subgroups that are identifiably at increased risk for breast cancer. Targeting the use of mammography to these groups could reap nearly all of the benefit and save everyone else a lot of bother.

  5. jrc says:

    “But still, no visible difference at all…”

    Maybe we should replace p-values with “visible differences”. Like “The difference between the two groups was 0.1sd (visible) for men wearing green, but only 0.01 (invisible) among women wearing red. Differences between men and women wearing purple were only visually apparent over small ranges of temperature when zoomed in really close, and are invisible at the macro-study level. There is no visible effect of differences among orange-wearing men and women (N=843,927), except on rainy days that are also bank holidays or hot days that are also religious holidays, in which case the difference is clearly visible (N=7).”

  6. Paul Alper says:

    Perhaps it needs to be emphasized that the above graphs come from a randomized control trial over a period of 25 years. Forget about the p-values and statistical significance; with the numbers given in the bottom graph, assuming that with mammography is better than without, the number needed to treat, NNT, is over 7000. The number needed to harm, NNH, while unobtainable from the graph, is intuitively low because the control arm is not invasive and mammography often begets unnecessary further medicalizing.

  7. Llewelyn Richards-Ward says:

    I guess, obviously, the impact of the screening process varies across nations due to inherent healthcare assumptions and practices. Given the one-size-for-all (egalitarian and fully state-funded) healthcare system, standardised measurement of longitudinal outcomes (in NZ, we are only 4M people), our mostly social-care approach, etc etc, well-conducted research here can and does provide some interesting insights into these types of healthcare systems. For example, https://www.nsu.govt.nz/system/files/resources/bsnzmortcohcceval_final7_8_dec.pdf. Here, we have a very ‘conservative management’ approach to most conditions. Hence, screening really is triage in order to ensure treatment efficacy and who best to match to Pharmac (they buy and manage medication, reducing big PHARMA profits) funded medications. So the basic idea to managing health-costs is to screen where and to whom it best works, then treat with effective interventions where there are very few ‘discretionary’ procedures.

  8. Elin says:

    Yes this is well known among people who read the research. Most women don’t get breast cancer and even fewer die from it. So overall at 25 years the breast cancer death cumulative probability is roughly .013. How much difference would you expect to see even if it is useful? A drop to .011? .010? lower?

  9. StronglyControlled says:

    Discussants would be wise to check the intervention in the Control arm.
    “…all women aged 50-59 in both arms received annual physical breast examinations.”
    Perhaps “30 year old mammography was as good as an annual breast examination” might be a good title for this post?

  10. JJ says:

    I worked as a statistician on a project about population wide effects for health interventions.
    1.Found no effect for mammograms at the population level in my country.
    2.Was perplexed, so examined the literature.
    3.Spent a day on the internet, found 0 studies indicating advantage for mammograms, and a few indicating no effect.
    4.Was “very surprised” about 4 years ago.
    5.Am not very surprised that this evidence has been unheeded…
    Sigh.

  11. Keith O'Rourke says:

    The evaluation of screening is very tricky and an area where some statisticians unfortunately do more harm than good.

    This old post was on PSA screening – http://andrewgelman.com/2010/04/15/when_engineers/

Leave a Reply