This post is by Phil Price
A Washington Post article says “In the first study of its kind, researchers from Washington State University and elsewhere found a 14 percent greater risk of head injuries to cyclists associated with cities that have bike share programs. In fact, when they compared raw head injury data for cyclists in five cities before and after they added bike share programs, the researchers found a 7.8 percent increase in the number of head injuries to cyclists.”
Actually that’s not even an example of “how to lie with statistics”, it’s simply an example of “how to lie”: As noted on StreetsBlog, data published in the study show that “In the cities that implemented bike-share…all injuries declined 28 percent, from 757 to 545. Head injuries declined 14 percent, from 319 to 273 per year. And moderate to severe head injuries also declined from 162 to 119. Meanwhile, in the control cities that do not have bike-share, all injuries increased slightly from 932 to 953 per year — 6 percent.” There’s a nice table on Streetsblog, taken from the study(make sure you read the caption).
So the number of head injuries declined by 14 percent, and the Washington Post reporter — Lenny Bernstein, for those of you keeping score at home — says they went up 7.8%. That’s a pretty big mistake! How did it happen? Well, the number of head injuries went down, but the number of injuries that were not head injuries went down even more, so the proportion of head injuries that were head injuries went up.
According to StreetsBlog, University of British Columbia public health professor Kay Ann Teschke “attempted to notify Bernstein of the problem with the article in the comments of the story, and he was initially dismissive. He has since admitted in the comments that she is right, but had not adjusted his piece substantially at the time we published this post.” (I don’t see that exchange in the comments, although I do see that other commenters have pointed out the error).
To be fair to Bernstein, it looks like he may have gotten his bad information straight from the researchers who did the study: The University of Washington’s Health Sciences NewsBeat also says “Risk of head injury among cyclists increased 14 percent after implementation of bike-share programs in several major cities”. It’s hard to fault Bernstein for getting the story wrong if he was just repeating errors that were in a press release approved by one of the study’s authors!
But how do Bernstein, the Washington Post, the study’s author at University of Washington (Janessa Graves), and the University of Washington justify their failure to correct this misinformation? It’s a major error, and it’s not that hard to edit a web page to insert a correction or retraction.
[Note added June 18: When I posted this I also emailed Bernstein and the UW Health Sciences Newsbeat to give them a heads-up and invite comment. Newsbeat has changed the story to make it clear that the proportion of injuries that are head injuries increased in the bike share cities. They do not note that the number of head injuries decreased, and it looks like they forgot to correct the headline so it's still wrong. At least they acknowledged the problem and did something, although I daresay most readers of that page will still be misled. But it's no longer flat wrong. Except the headline.]
Of course, even simply retracting the story is a missed opportunity: the real story here is that injuries went down in bike share cities in spite of the fact that there were more people riding. That’s a surprise! As a bike commuter, I know that it has long been argued that biking becomes safer per biker-mile as more people ride, because drivers become more alert to the likely presence of bikes. But I would not have expected that the decrease in risk per mile would more than counteract the number of miles ridden, such that the number of injuries would go down. Or, of course, maybe that’s not what happened, maybe there were other changes that were coincident with the introduction of bike share programs, that decreased risk in the bike share cities but not the control cities.
This sort of thing — by which I mean mis-reporting of scientific results in general — is just so, so frustrating and demoralizing to me. If people think bike share programs substantially increase the risk of injury, that belief has consequences. It affects the amount of public support for such programs (and for biking in general) as well as affecting individuals’ decisions about whether or not to use those programs themselves. To see these stories get twisted around, and to see journalists refuse to correct them…grrrrr.
This post is by Phil Price
[Andrew, please add "Ethics" and "Journalism" categories to this blog]