Where’d the $2500 come from?

Brad Buchsbaum writes:

Sometimes I read the New York Times “Well” articles on science and health. It’s a mixed bag, sometimes it’s quite good and sometimes not. I came across this yesterday:

What’s the Value of Exercise? $2,500

For people still struggling to make time for exercise, a new study offers a strong incentive: You’ll save $2,500 a year.

The savings, a result of reduced medical costs, don’t require much effort to accrue — just 30 minutes of walking five days a week is enough.

The findings come from an analysis of 26,239 men and women, published today in the Journal of the American Heart Association. . . .

I [Buchsbaum] thought: I wonder where the number came from? So I tracked down the paper referred to in the article (which was unhelpfully not linked or properly named).

I was horrified to find that the $2500 figure appears to be nowhere in the paper (see table 2). Moreover, the closest number I could find ($1900) was based on a regression model without covarying age, sex, ethnicity, income, or anything else. Of course older people exercise less and spend more on healthcare!

I sent the following email (see below) to the NYTimes author, but she has not responded.

At any rate, I thought this example of very high-profile science-blogging to be particularly egregious, so I thought I’d bring it to your attention.

The research article is Economic Impact of Moderate-Vigorous Physical Activity Among Those With and Without Established Cardiovascular Disease: 2012 Medical Expenditure Panel Survey, by Javier Valero-Elizondo, Joseph Salami, Chukwuemeka Osondu, Oluseye Ogunmoroti, Alejandro Arrieta, Erica Spatz, Adnan Younus, Jamal Rana, Salim Virani, Ron Blankstein, Michael Blaha, Emir Veledar, and Khurram Nasir.

And here’s Buchsbaum’s letter to Gretchen Reynolds, the author of that news article:

I very much enjoy your health articles for the New York Times. Sometimes I try and find the paper and examine the data, just for my own benefit.

After perusing the paper, I’m was not quite sure where the $2500 figure came from. In table 2 (see attached paper), the unadjusted expenditures are reported over all subjects.

non-optimal PA: $5397, optimal PA: $3443 for a difference of $1900.

This is close to $2500 but your number is higher.

However, remember, this is an *unadjusted model*. It does not account for age, sex, family income, race/ethnicity, insurance type, geographical location or comorbidity.

In other words, it’s a virtually useless model.

Lets look at Model 3, which does account for the above factors.

non-optimal PA: $4867, optimal PA: $4153 for a difference of $714

So $714 closer to the mark.

BUT, this includes ALL subjects, including those with cardiovascular disease (CVD).

If you look at people without CVD then the estimates depend on the cardiovascular risk profile (CRF). If you have an average or optimal profile then the difference is around $430 or $493. If you have a “poor” profile, then the difference is around $1060 (although the 95% confidence intervals overlapped, meaning the effect was not reliable).

What is my conclusion?

I’m afraid the title of your article is misleading since it is larger (by $600) than the $1900 estimate based on the meaningless unadjusted model! Even if the title was “What’s the Value of Exercise? $700”, it would still be misleading, because it implicitly assumes a causal relationship between exercise and expenditure.

Remember also that the adjusted variables are only the measures the authors happened to record. There are dozens of potentially other mediating variables which are related to both physical exercise and health expenditures. Including these other adjusting factors might further reduce the estimates.

Best Regards,

It’s just a news article so some oversimplification is perhaps unavoidable. But I do wonder where the $2500 number came from. I’m guessing it’s from some press release but I don’t know.

Also, I’m surprised the reporter didn’t respond to the email. But maybe New York Times reporters get too many emails to respond to, or even read. I should also emphasize that I did not read that news article or the scientific paper in detail, so I’m not endorsing (or disagreeing with) Buchsbaum’s claim. Here I’m just interested the general challenge of tracking down numbers like that $2500 that have no apparent source.

29 thoughts on “Where’d the $2500 come from?

  1. Even if you learned the source, would the number $2500 become any more meaningful? Depends on how much faith you put in the model and its assumptions. This seems to be part of Buchsbaum’s argument about the implicit assumption of “a causal relationship between exercise and expenditure”.

    The claim that exercise saves some amount of money seems reasonable (maybe obvious), but putting an exact figure, whether $700, $2500, or otherwise, is bound to be misleading and sensitive to assumptions regardless of how it was derived.

  2. It’s not clear to me that the adjusted estimate is any more (causally) meaningful than the unadjusted estimate, given the possibility of residual confounding. Trivial point, but even if we have just two confounders, conditioning on one but not the other might move our estimate further away from the relevant causal parameter, even if we’ve correctly specified the conditional distribution.

  3. Interestingly, the annual savings figure of $2500 coincides with the annual opportunity cost of daily walking (0.5 x 250 x $20.00@) assuming a time value of $20 per hour which is fairly standard in the literature.

    • Interesting. And, of course, subject to the usual caveats about opportunity cost calculations. (It’s not that they’re wrong or shouldn’t be done, it’s just that it’s clear humans don’t operate that way.)

      I’ve also seen a calculation that for every hour you spend bicycling, on average you increase your lifespan for about an hour, so it’s as if cycling takes up no time at all.

      Math is a wondrous thing!

      • Ah, but even if the one hour of bicycling giving one hour of life were true, it would be a bad deal unless you enjoy cycling (which I would guess you do). Effectively, you are exchanging one hour in your 30s for one hour in your 80s. I know which one I would rather have!

        • Reminds me of a friend who, when told that every cigarette takes seven minutes off your life, always replied: “But those are the minutes at the end.”

        • But the end of life for a smoker is much worse then the end of life for a non-smoker (as someone with a grandfather who died from emphysema).

        • If you know a lot of people retirement age and above, you see that there is a huge variability in how healthy they are, and therefore what they can do.

          A huge fraction of this variability is luck — genetic luck, and just plain luck. But there’s a portion of that variability under our control through health habits — good habits and bad habits.

          Overall, habits like exercise, not smoking, and keeping your weight under control add better quality of life in your 70s and beyond.

          “The race is not always to the swift, nor the blood sugar level to the thin, but that’s the way to bet.” (paraphrasing Hugh Keough)

      • “Interesting. And, of course, subject to the usual caveats about opportunity cost calculations. (It’s not that they’re wrong or shouldn’t be done, it’s just that it’s clear humans don’t operate that way.)”

        GS: What claim, exactly, are you making when you say “…it’s just that it’s clear humans don’t operate that way.”

    • I’m going to go out on a limb and say that this is straight up the major reason for lack of exercise, we’re paying people to sit in front of computers or in the cabs of trucks and do things. When you add in travel time to a gym or the time it takes to maintain your bike or take a shower or whatever, the opportunity cost of exercise is too high for many people. The median black family has less than $7000 in total net worth (I have this data from the Census). Telling them to get a job where they can take an hour off a day and do exercise is truly insensitive to their situation. But it’s not just communities that are marginalized, the same is true for many people well into the middle class. Being middle class entails committing to large housing cost obligations. I’ve read surveys where a shocking number of people have no savings that would cover a $1500 car repair bill, and they’d have to put that on a credit card.

      It would shock me if my preferred method of solving all these problems: the UBI didn’t also generally improve health by decoupling survival from working like a dog for stagnating or declining wages.

      • The median black family has less than $7000 in total net worth (I have this data from the Census). Telling them to get a job where they can take an hour off a day and do exercise is truly insensitive to their situation.

        A few years ago I read a warning from the Surgeon General on something and wondered “who is the surgeon general?”. Then I looked it up and found that a current top priority was figuring out how to deal with the problem that exercise messes up your hair:

        There are studies that show that when we ask women – and particularly African-American women – why don’t you exercise? And they would say, well, I just spent a lot of money on my hair, I don’t want to sweat(ph) my hair back. I ask a number of my older patients – my older, white patients; they say the same thing: I’m not going to go messing up my hair when I came from the beauty parlor. So we decided to try to at least get rid of one barrier – find an exercise-friendly hairstyle.

        http://www.npr.org/2012/08/08/158419580/surgeon-general-dont-let-hair-get-in-the-way

        • I have anecdotal evidence that black women in the US tend not to breast-feed because they tend to believe it will lead to sagging breasts… Obviously this wasn’t a scientific poll or anything, but I do think we have a LONG way to go towards raising the median education level about biological science and its implications for health, across all races and socioeconomic classes. And all this bullshit waffling hype in the news about “eating X increases risk of Y” and then a week later “not eating X increases your risk for Y” and soforth, that just harms the societal interests towards improving people’s general health because it sends a strong signal “THIS FIELD IS FULL OF BULLSHIT”.

        • > I have anecdotal evidence that black women in the US tend not to breast-feed because they tend to believe it will lead to sagging breasts…

          Are you implying that breast-feeding will not result in increased sagginess?

        • No, I’m implying that this effect is relatively un-studied, and all the evidence there was anecdotal, and the women involved had basically no knowledge of the health benefits to their children from breastfeeding. They were making an extremely low-information decision on the basis of what was essentially an aesthetic consideration along the lines of the hair issue. My assumption is that if they understood more about the whole decision they would probably make a different one.

        • A similar thing seems to occur in immigrant populations from Latin America. The grandmother who grew up in say Honduras tells the new parents to stuff their children with extra calories (adding rice cereal to formula for example) because that was actually good advice in “the old country” where food was more scarce and bouts of diarrhea resulted in high infant mortality among thin childrean, but here in the US it doesn’t apply. Again, lack of correct assumptions and knowledge, together with cultural heuristic/superstitious reasoning leads to potentially bad health outcomes.

  4. > Because they wanted to look at costs related to inactivity, the researchers decided to focus on expenses among this group related to cardiovascular disease, since its incidence and severity are known to be affected by whether someone exercises. […] On average, someone who met the exercise guidelines paid $2,500 less in annual health care expenses related to heart disease than someone who did not walk or otherwise move for 30 minutes five times per week.

    My bet is that this is a reference to the CVD group in table 2. For model 3, expenditures are $12660 and $10092 for those with non-optimal and optimal PA resulting in a difference of $2567.

    • Chris Hardwick implied the same thing on his comedy show @midnight last week. I would very much like to know how the question was phrased. I have seen many asinine survey questions that just beg to be pranked, and I guarantee that if a condescendingly worded question that EVERYBODY knows the answer to were presented to my circle of friends there would be fisticuffs to compete for the most absurd answer. I would wager that if the adults included college students, one could demonstrate from that subset that fifty percent of college students believes chocolate milk comes from brown cows and the other fifty percent believes it comes from rusty pipes. As Doctor House says, people lie. They do it all the time just for the fun of it. What troubles me more than anything is the other 93% with no sense of humor.

    • It does come from brown cows, just like milk without added flavoring. It comes from other colors of cows as well. Although of course color is just a social construct of dubious origin.

      It is astonishing that 93% would think no brown cows were involved in the production of chocolate milk.

      • Regarding that conspiracy theory + “agw skeptic” story at your link, since then it was revealed that another author of that paper was impersonating prominent “skeptics” on internet forums:

        John Cook: Sorry about the Lubos thing. Was posting some Lubos comments for the UWA experiment and forgot to log back in as John Cook.

        http://motls.blogspot.com/2015/07/identity-theft-thief-of-lubosmotl-turns.html

        I’m not sure if that affected those results but I guess you also have to worry about pollsters purposefully filling out their own polls to get “sexy” results or whatever. For example no one would care about that milk poll if not for that chocolate milk result.

  5. Buchsbaum says in his letter:
    “If you have a “poor” profile, then the difference is around $1060 (although the 95% confidence intervals overlapped, meaning the effect was not reliable).”
    This is not necessarily true. It is possible for two univariate CIs to overlap even though the two parameters are significantly different. We need to look at the CI for the difference in the two parameters to determine if they are significantly different.

  6. Yea, I don’t like when numbers are invented out of thin air. I also don’t like this attempt to marginalize over everything important to come up with an ‘average’ benefit of exercise. The benefits are going to vary hugely between individuals depending on their genetics, health history, current and future lifestyle context (diet, smoking, etc.), etc. Also, on principle, we need to be measuring physiological outcomes of fitness (strength, lean body mass/fat mass ratio, VO2 max) and testing exercise programs that efficiently deliver those- getting up and moving around is important, but there’s good reason to think that most of the benefits from exercise are gonna be collinear with ability to actually create physiological adaptations (i.e. http://www.onlinejacc.org/content/65/6/629)

  7. Two things strike me with these type of articles. First, it seems that there are cultural drivers for the metric used, in this case, money. In NZ and UK, the benefits and costs of exercise are generally discussed in the media in terms of health and longevity, by comparison. The Benthamite econometric view always seems rather a dispassionate and unhelpful approach, to me at least (a lot of my time is spent working with motivating people to rehabilitative health goals).

    Second, media sources with a reputation of eminence and believability can create a dogma that is ‘known to be true’, or as often is said ‘Well, we know…” or ‘Of course, studies show…’, followed by the appropriate health advice. An example of this is the Whitehall prospective study https://en.wikipedia.org/wiki/Whitehall_Study. In NZ & Aus, exercise is typically ‘prescribed’ as ideal if it is for around 3.5 hours in total, over 5 or so days. Over the years, I often ask colleagues about where they get this ‘number’, typically with the ‘multiple studies’ fogged reply. IF one ever bothers to read the study, it is probably fair to say that the guidelines are not unhelpful. However, what one doesn’t say to patients, which is from the original study, is that social class was a powerful predictor of mortality and one might argue that improving one’s social position would have a greater benefit!! So, while saying that one can save $2500 from exercise may be motivational, much like some eminent pizza studies, :(, I’m not sure this ever should be seen as a source of science or health advice, for all the reasons noted above. So what ought to be the standards for reporting science in the press?

  8. “You’ll save $2,500 a year.”

    As if the data say anything meaningful about individuals. Of course the study can’t be done “single-subject,” and the information gleaned from the study presented here might be OK for certain purposes, but if you are interested in the question from the standpoint of individuals, it would be better to estimate the savings from a few dozen individuals by looking at measures of cardiovascular functioning etc. as a function of exercise, and those variables can be looked at with SSDs. The estimates, of course, would depend on other data concerning “turning cardio function into savings” but, at least, one could start exploring some avenue by which some meaningful estimate for an individual could be obtained.

    • Yes! Although I would argue that with e.g. the article I linked above, where we have prospective estimate of male mortality as a function of cardiorespiratory fitness (and controlled for most main risk factors), we could more readily model the value of exercise based on fitness achieved (measured by METS/VO2max), rather than how long folks were waving their arms around with 2-lb dumbbells or something else dumb.

Leave a Reply to Jack PQ Cancel reply

Your email address will not be published. Required fields are marked *