Skip to content
 

“The graph clearly shows that mammography adds virtually nothing to survival and if anything, decreases survival (and increases cost and provides unnecessary treatment)”

Paul Alper writes:

You recently posted on

graphs and how to convey information.  I don’t believe you have ever posted anything on this dynamite randomized clinical trial of 90,000 (!!) 40-59 year-old women over a 25-year period (also !!). The graphs below are figures 2, 3 and 4 respectively, of
The control was physical exam only and the treatment was physical exam plus mammography.
F2 BCmortality.png
F3 BCspecific.png
F4 BCdiagnosis.png
The graph clearly shows that mammography adds virtually nothing to survival and if anything, decreases survival (and increases cost and provides unnecessary treatment).  Note the superfluousness of the p-values.
   There is an accompanying editorial in the BMJ
which refers to “vested interests” which can override any statistics, no matter how striking:

We agree with Miller and colleagues that “the rationale for screening by mammography be urgently reassessed by policy makers.” As time goes by we do indeed need more efficient mechanisms to reconsider priorities and recommendations for mammography screening and other medical interventions. This is not an easy task, because governments, research funders, scientists, and medical practitioners may have vested interests in continuing activities that are well established.

And for the aging males in your audience:

Nevertheless, the UK National Screening Committee does recommend mammography screening for breast cancer but not prostate specific antigen screening for prostate cancer.
Because the scientific rationale to recommend screening or not does not differ noticeably between breast and prostate cancer, political pressure and beliefs might have a role.

Pretty amazing that the effect is 0. That’s hard to believe! Although I guess the point is that a thorough physical exam won’t miss much.

26 Comments

  1. Anonymous says:

    For those looking at the blog who haven’t yet clicked the journal article (it’s open access at least to me). The graphs are:

    1) All Cause Mortality
    2) Breast Cancer Mortality
    3) All Cancer Mortality.

    There’s the slightest hint of an indication in the 3rd graph that maybe the exposure to x-rays could even increase all-cancer mortality (with mammography arm consistently slightly below the control arm). Too noisy to distinguish such a small effect reliably though.

  2. jonathan says:

    My dad was a radiologist. He believed the main problem with mammography was that it had become a substitute for screening. I don’t know if it affected the incidence of women self-checking at home – remember the commercials on TV about that? – but he felt it substituted for professional physical exam. And the problem with that was simple: more cancers would be caught later – put off by cost, discomfort, time needed for mammography – and of these some would be aggressive. He also felt strongly that mammography made people believe some things about cancer that were/are wrong: that a clean mammogram meant you could go without for a while. His feelings on this were shaped, I think, by the risk to radiologists from lawsuits. Have a mammogram, don’t do a check-up or maybe even a self-check, find cancer, extend chain of blame back to radiologist.

    As a non-doctor, I look at that process as being fairly typical: a technology substitute is presumed to be better, displaces more processes than it actually should, and then is pressured to perform beyond its capacity when failures become obvious. Or in human terms, expect mammography to be magic, get sick, experience great pain and suffering, maybe die and then pursue money back through the chain … when the actual fault is systemic belief that shifted the medical process away from actual best practices. We see this kind of thing all the time. Remember when the Corps of Engineers straightened waterways because they could? The best practice would have been to mirror twists and turns, as dredging had for centuries, because river life functions far better in the conditions of a twisting and turning waterway. The technological solution displaced the best practice and we ended up with lifeless, polluted waterways that also tended to erode more in unfortunate ways.

  3. Jonathan Gilligan says:

    It’s worth noting that the control arm received one physical exam at the beginning, no annual physical exams during their 40s, and then annual physical exams from age 50-59, so for women in their 40s the difference is NOT due to annual physical exams being as good as mammograms and seems to me to be more an indication that early detection of cancer has no clinically significant benefit.

  4. Picking out only the outlier Canadian study (or studies – generally dealt with as two) is a biased way of looking at this question. Note they only had 5 annual mammograms: speaking of it as though this was studying long-term screening, as opposed to being long-term follow-up of a relatively short period of screening, is misleading.

    Anyone interested in a fuller picture of the evidence on breast screening with mammography can see the major studies of effectiveness here in my comments at PubMed Commons: http://www.ncbi.nlm.nih.gov/myncbi/hilda.bastian.1/comments/ – and I wrote a blog post about cognitive/statistical bias and mammography this week at Scientific American: http://blogs.scientificamerican.com/absolutely-maybe/2014/05/01/a-viewers-guide-to-mammography-evidence-ping-pong/.

    • Andrew says:

      Hilda:

      In your linked post, you wrote, “mammography triggers ardent ideological opposition.” I had not heard about this. What is the ideological opposition to mammography and who has expressed it?

  5. question says:

    When was the last medical treatment devised for which there is unequivocal evidence of usefulness? Not in terms of proxy measures (eg, lowers cholesterol), but in actually reducing incidence or prolonging survival substantially. I don’t mean for this to be a sarcastic/rhetorical question, just looking to collect evidence of successful research.

    • I’ve got to imagine that there are ALL SORTS of surgical techniques which are unequivocally useful, and recent. Everything from skin grafts for burns to facial transplants to certain vascular surgery techniques to bone reconstruction techniques etc. Most of the outcomes are likely to be things like “can chew again” or whatever. Outcomes that are so obvious they aren’t even mentioned as topics of study.

      Also, in certain diseases there are definitely vastly improved treatment techniques. The current treatment for lymphoma is *substantially* better than the standard treatment of say 20 years ago (or so I’m told), I think the improvement is so large it’s well outside the statistical noise range.

      • Another example might be something like cataract surgery. The modern artificial lenses are technologically superior in almost every way, and the incidence of complications has become pretty darn low. I know several people who’ve had cataract surgery and it’s been transformative, their eyesight is better than it ever was during their entire life, simultaneously correcting the cataracts and myopia and astigmatism.

      • question says:

        Do you have a source I can start at for the claim about lymphoma? Also, how much are surgical techniques informed by theories told to us by “basic science”? I am not familiar with that literature but I would imagine much less so that pharmacological approaches.

    • Thomas says:

      I’ve found Gilbert Welch’s work on this very illuminating. His standard example of a more or less unequivocally useful screening + treatment operation was for hypertension (high blood pressure). He talks about it in THIS talk.

      • Paul Alper says:

        Thomas is referring to a 99 minute presentation which Welch gave a couple of years ago and thus, precedes the large Canadian study. I cannot recommend highly enough this video which is absolutely fascinating. Likewise his two books.
        Thus far on the blog, no one has commented on the accompanying BMJ editorial, http://www.bmj.com/content/348/bmj.g1403 which deals with the vested interests of mammography. The NYT article claimed that 37 million(!) mammograms are done per year in the U.S. at $100 each. With that kind of money at stake, the “mammography lobby” is not about to quietly disappear despite repeated clinical trials which indicate mammography’s lack of efficacy (for women under 60).
        In fact, from a 2011 Cochrane review, http://www.cochrane.org/news/blog/mammography-screening-ten-years-reflections-decade-2001-review, Peter Gøtzsche, Director of the Nordic Cochrane Centre says,
        “Our finding of increased mastectomies has consistently been ignored by screening advocates for 10 years, and information from many cancer charities and governmental agencies continues to state the opposite – that screening decreases mastectomies – despite having no reliable data to support this claim.”
        It is interesting to contrast mammography screening holding on to the status quo with the complete turnaround regarding morcelators used to remove fibroid tumors, http://qualityhealthcareplease.wordpress.com/2013/12/21/dangers-for-removing-fibroids-the-morcelator/.

      • I vaguely remember how there was just a recent review study about how controlling blood pressure was essentially useless in people under… maybe 50 years old, and how it had very little effect even in those groups where it did have a definite effect…. This might have been linked from this blog a few months back.

        I will check into your linked presentation and also look for the review article if I can find it.

        • Thomas says:

          What I really like about Welsh is that he has these clear examples (at least from his point of view) of where it works (hypertension), where it does harm (PSA), and where it’s a close call (mammography). I like his style, you might say. Of course, if he’s wrong about any particular case then he’s wrong. I just like the way he makes the case.

  6. Just because a study is large doesn’t mean that its findings are necessarily true.

    • Norman says:

      This is true, but the point is that a large study, if done well, should pick up even small effects if they exist. A small study, even if done well, could miss small effects. As is always the case, an unusual sample + selection bias can produce false results, but in practice (and in keeping with Ioannidis’s 2005 paper) the selection bias is in favor of finding apparent effects where none truly exist, rather than the other way around.

      Do you believe the study was poorly designed for the research question asked? If so, what criticisms of the actual study do you have? This would be a more helpful contribution than to argue on a statistically-oriented blog that we face uncertainty.

      • mpledger says:

        I guess I’d be worried that the women who missed out on free mammograms might have gone and got them through their “usual care” provider.

        I’d be worried that the physician doing the physical exam would do it with extra rigor since they don’t want to look bad by not detecting cancer (and they don’t know which arm their patient is in).

        I’d be worried that patients detected earlier would get inferior treatment to patients detected later i.e. cancer treatments improved as a faster rate than early stage cancer harmed people.

  7. K? O'Rourke says:

    This statement does concern me “We used Cox proportional hazards model to calculate hazard ratios with 95% confidence intervals.”

    Now, I did work with the first author on a liver cancer screening study in 1990,s and we did discuss the lag effect extensively (which implies the hazards cannot be proportional), so maybe they have ruled that out as being an important part of the model to be wrong about (likely the case here).

    Still I would want another group of statisticians who have expertise in screening studies to closely review and ideally re-analyse the data.

    Some background on why screening trials are hard for most statisticians to grasp is here http://andrewgelman.com/2010/04/15/when_engineers/#comments

  8. John P says:

    “Orac” is the nom de blog of an oncologist who primarily treats breast cancer in his clinic. He blogs at http://scienceblogs.com/insolence. Orac has discussed mammography screening in many posts on his blog. You can search for mammography there if you’re curious.

    This is a complicated issue. There is no way I could do it justice in a comment. I’m not qualified, anyway (I’m a physicist). Some of the terms that come up are overdiagnosis, overscreening, lead time bias, and length time bias. I agree with Orac that mammography is oversold and of questionable utility for women under 60. There is a lot of money involved in screening for breast cancer.

    Orac discussing early detection, lead time bias, and length time bias: http://scienceblogs.com/insolence/2007/04/02/detecting-cancer-early-part-1-more-compl/

    A post about overdiagnosis and mammography, with the above graphs included:
    http://scienceblogs.com/insolence/2014/02/17/the-canadian-breast-screening-study-attacked-why-do-doctors-have-such-a-hard-time-with-the-concept-of-overdiagnosis/

  9. DocinPA says:

    This study is notoriously crappy. Crappy films, crappy radiologists, selection biases, the list is long. One of the authors of the BMJ article even denounced this trial several years ago, ie., they were against it before they were for it. And here is Dr. Laszlo Tabar’s demolition of this whole sorry effort:

    http://www.mammographyed.com/News/NewsDetail.aspx?a=8076

  10. TheWo says:

    Gosh, Tabar has many undisclosed competing interests and has notoriously spread distortions about mammogram-dissenting studies. This is not exception. Read thru the comments pertaining to this Canadian study at the BMJ and you’ll get to recognize that Tabar is a hack for the mammogram industry. For an accurate description of mammography I’d suggest you read “Mammography Screening: Truth, Lies and Controversy” (2012) by Peter Gøtzsche, a mammogram scientist, and “The Mammogram Myth” (2013) by Rolf Hefti, an independent investigator.

Leave a Reply