There have been several news stories about a recently completed study of the effects of a low-fat diet on heart disease, colon cancer, and breast cancer. One of those stories is here. Some excerpts:
The large primary prevention trial conducted at 40 U.S. clinical centers from 1993 to 2005 enlisted 48,835 postmenopausal women ages 50 to 79 years.
Women in all three trials were randomly assigned to an intervention group (40%, n=19,541) with intensive behavior modification sessions and the goal of reducing fat intake to 20% of energy and increasing vegetable and fruit intake to at least five servings a day and grains to six servings.
Participants in a comparison group (60%, n= 29,294) were not asked to make changes…
So far, so good: we know the sample sizes, we know something about the interventions. What about the results? The article goes on to say:
Over an 8.1-year average follow-up, a low-fat diet did not lead to a significant reduction in invasive breast cancer risk, the researchers reported.
The number of women (annualized incidence rate) who developed breast cancer was 655 (0.42%) in the intervention group and 1,072 (0.45%) in the comparison group (hazard ratio 0.91; 95% confidence interval 0.83-1.01).
For starters, 655 cases out of 19541 women in the intervention group is 3.3%, not 0.42%, so I’m not sure quite what is going on with the numbers. Presumably something got lost in translation from the scientific publication to the news article.
But the thing that really strikes me is that the 95% confidence interval for breast cancer prevention just barely includes 1.00, or no effect. So the news stories all say some variation of “there is no significant benefit” from a low-fat diet. I suspect, though of course we will never know for sure, that if there had been just a couple fewer cases in the intervention group, or a couple more in the comparison group, so that the 95% confidence interval topped out at 0.99 rather than 1.01, the news articles would have trumpeted “Significant reduction in breast cancer from a low-fat diet.” This is an issue Andrew touched on a few weeks ago on this blog.
Like a lot of people who perform statistical analyses, I’ve always been unhappy with “statistical significance” (the term, and the concept) for two reasons: (1) in common speech, “significant” means “important”, but that is not true in stat-speak, where a parameter value can be “statistically significant” even though it is of no practical importance, and can be of great practical importance even if it is not “statistically significant” (the latter being the case if sample sizes or other sources of uncertainty are so large that even important effects can be missed); and (2) when did God say that 95% is the threshold for “statistical significance”? What’s so magic about 1/20, as opposed to 1/18 or 1/21 or anything else for that matter?
In the current study, there is mild evidence for a small effect on breast cancer; the best guess (from this study alone) would suggest that a low-fat diet, of the type they tested, reduces breast cancer incidence by something between 5% and 15%, although larger or smaller benefits (or even a small penalty) cannot be ruled out. I wish reporters would put it that way, rather than declaring that the result did or did not meet some arbitrary (though admittedly customary) “significance” standard.