Skip to content
 

As if we needed another example of lying with statistics and not issuing a correction: bike-share injuries

This post is by Phil Price

A Washington Post article says “In the first study of its kind, researchers from Washington State University and elsewhere found  a 14 percent greater risk of head injuries to cyclists associated with cities that have bike share programs. In fact, when they compared raw head injury data for cyclists in five cities before and after they added bike share programs, the researchers found a 7.8 percent increase in the number of head injuries to cyclists.”

Actually that’s not even an example of “how to lie with statistics”, it’s simply an example of “how to lie”: As noted on StreetsBlog, data published in the study show that “In the cities that implemented bike-share…all injuries declined 28 percent, from 757 to 545. Head injuries declined 14 percent, from 319 to 273 per year. And moderate to severe head injuries also declined from 162 to 119. Meanwhile, in the control cities that do not have bike-share, all injuries increased slightly from 932 to 953 per year — 6 percent.”  There’s a nice table on Streetsblog, taken from the study(make sure you read the caption).

So the number of head injuries declined by 14 percent, and the Washington Post reporter — Lenny Bernstein, for those of you keeping score at home — says they went up 7.8%.  That’s a pretty big mistake! How did it happen?  Well, the number of head injuries went down, but the number of injuries that were not head injuries went down even more, so the proportion of head injuries that were head injuries went up.
According to StreetsBlog, University of British Columbia public health professor Kay Ann Teschke “attempted to notify Bernstein of the problem with the article in the comments of the story, and he was initially dismissive. He has since admitted in the comments that she is right, but had not adjusted his piece substantially at the time we published this post.” (I don’t see that exchange in the comments, although I do see that other commenters have pointed out the error).

To be fair to Bernstein, it looks like he may have gotten his bad information straight from the researchers who did the study: The University of Washington’s Health Sciences NewsBeat also says “Risk of head injury among cyclists increased 14 percent after implementation of bike-share programs in several major cities”. It’s hard to fault Bernstein for getting the story wrong if he was just repeating errors that were in a press release approved by one of the study’s authors!

But how do Bernstein, the Washington Post, the study’s author at University of Washington (Janessa Graves), and the University of Washington justify their failure to correct this misinformation?  It’s a major error, and it’s not that hard to edit a web page to insert a correction or retraction.

[Note added June 18: When I posted this I also emailed Bernstein and the UW Health Sciences Newsbeat to give them a heads-up and invite comment. Newsbeat has changed the story to make it clear that the proportion of injuries that are head injuries increased in the bike share cities. They do not note that the number of head injuries decreased, and it looks like they forgot to correct the headline so it's still wrong. At least they acknowledged the problem and did something, although I daresay most readers of that page will still be misled. But it's no longer flat wrong. Except the headline.]

Of course, even simply retracting the story is a missed opportunity: the real story here is that injuries went down in bike share cities in spite of the fact that there were more people riding. That’s a surprise!  As a bike commuter, I know that it has long been argued that biking becomes safer per biker-mile as more people ride, because drivers become more alert to the likely presence of bikes. But I would not have expected that the decrease in risk per mile would more than counteract the number of miles ridden, such that the number of injuries would go down. Or, of course, maybe that’s not what happened, maybe there were other changes that were coincident with the introduction of bike share programs, that decreased risk in the bike share cities but not the control cities.

This sort of thing — by which I mean mis-reporting of scientific results in general — is just so, so frustrating and demoralizing to me. If people think bike share programs substantially increase the risk of injury, that belief has consequences. It affects the amount of public support for such programs (and for biking in general) as well as affecting individuals’ decisions about whether or not to use those programs themselves. To see these stories get twisted around, and to see journalists refuse to correct them…grrrrr.

This post is by Phil Price
[Andrew, please add "Ethics" and "Journalism" categories to this blog]

26 Comments

  1. John Mashey says:

    Misleading statements are bad enough, but the general topic is certainly worthy of statistical investigation.
    Anecdotal, from experience around SF Bay area:

    1) Bike usage seems to follow a logistic S-curve:
    a) A Long period when the only cyclists are really dedicated ones, are good cyclists, wear helmets, ride defensively, but there are few if any bike lanes, and motorists are often unfamiliar with them. I once read a blog by a guy who commuted to work by bike .. in Los Angeles. Did not sound like fun.

    b) Bike usage grows to the point where some critical mass occurs, bike clubs grow, more towns put in bike lanes when redoing roads, more casual riders start riding, more bike stores open and there’s a real inflection point in growth.

    c) Many more people bike, and this is a zone where I might worry about accidents, as it is all too easy for people to jump on bikes without quite knowing how to be careful. San Francisco has a very strong cycling community … but sometimes, tourists on rental bikes can be a bit scary.
    Sometime in here, towns start offering training classes for kids, a good thing … as I’ve seen cases where some parents decided to get back to cycling after decades because they had kids of the right age, and they’d sometimes not be careful enough.
    Bike lanes become pervasive … and even separate bike paths that allow pretty good mobility in an area, including bike bridges over freeways. But still, there are accidents.

    d) Then, in some places, like some in the Netherlands, and on some college campuses, cycling becomes the default.

    Anyway, there is probably room for some good statistical work to relate outcomes with the local intensity of cycling.

  2. Chip Lynch says:

    It looks like they changed the title to be “proportion”, but that seems like the only change required. I’m not sure I see this as egregious… the problem is that they don’t seem to have information on overall ridership, so in my opinion both sides are incomplete on this, while neither is really wrong. It probably should not have been published due to this oversight, but a general news journalistic post about a scientific journal article gets to share some blame, and I’d point my finger more at the original article in this particular case.

    The number we’re interested in should be total percentage of cyclists with a) any injury and b) head injuries. Since we don’t have total number of cyclists (unless it’s somewhere I missed) we can’t calculate those rates. Knowing that the number of total accidents went down is interesting, knowing that the percentage of head injuries to total injuries went up is interesting, and both points could probably inform decision making. But both are incomplete.

    • Phil says:

      Chip, they said “the researchers found a 7.8 percent increase in the number of head injuries to cyclists” but in fact the number of head injuries to cyclists declined 14%! I don’t know how you can say the claim isn’t “really wrong”. It is really wrong.

      What you’re saying is that what they should be looking is at head injuries to cyclists divided by the number of cyclists, or perhaps divided by the number of bicycle-miles-traveled or something. I agree that that would be better. Indeed, if the number of head injuries had gone up a lot, but so had the number of bicycle miles traveled, one could argue about what is the right or wrong way to look at the numbers. Then we might be having a “damn lies and statistics” discussion. But that’s not the case here, they’re saying a number went up 7.8% when in fact it went down 14%. That is indeed really wrong.

  3. jrkrideau says:

    Well, perhaps I’m letting my old prejudices get the better of me but the name Rivara combined with the “Harborview Injury Prevention and Research Center” make me immediatel suspicious. Rivara was one of the authors of the 1989 NEJM paper (Thompson, Rivara and Thompson, 1989) that came up with that “80% of head injuries are prevented by helmets” meme. I can only assume that they want to persuade people to wear helmets no matter what. TRT seem to follow the “Don’t confuse me with facts, I KNOW helmets work” approach.

    Re “I would not have expected that the decrease in risk per mile would more than counteract the number of miles ridden, such that the number of injuries would go down.” you might want to have a look at

    Jacobsen, P. L. (2003). Safety in numbers: more walkers and bicyclists, safer walking and bicycling. Injury Prevention, 9(3), 205–209.

    Somewhere in there, I think he talks about a non-linear trend—I have read two abstracts of the paper and one mentions it and one does not and it’s been years since I read the paper.

  4. stuart says:

    Andrew’s conjecture that it is something coincident with bike share that’s causing the decline in injuries makes sense here in Chicago. We have what is planned to become one of the biggest bike-share programs anywhere (www.divvybikes.com). At the same time (but independently, perhaps) the city started putting in protected bike lanes (http://www.activetrans.org/bikeways/101), and the mayor plans for 100 miles of these.

    • zbicyclist says:

      This is potentially separable; the protected bike lanes started in specific areas of the city (e.g. Dearborn St), so one could do a geographic analysis of the injuries.

      (also thanks to jrkrideau for his post. I think some of the studies which show big helmet effects also used the proportional injury methodology, but I haven’t felt the need to review the helmet literature in some years.)

    • Phil says:

      Man, what does a guy have to do to get credit for a blog post?

      Yes, bike lanes are a good example of what might have changed in bike share cities in addition to instituting bike share. Presumably the city could choose between (do nothing, add bike share but not bike lanes, add bike lanes but not bike share, add both), and if most bike share cities choose the latter then it’s hard to tell how much to credit bike share vs bike lanes.

      And there’s even the possibility that whatever changed has nothing to do with bike policy or actions at all, maybe it’s weather-related or something. The control group and intervention group of cities are both small.

    • Calum says:

      Anecdotally, my view is that motorists learn very quickly that bike-share users are extremely poor cyclists with zero road sense or awareness. Motorists respond by treating them as if they will do the most dangerous thing possible at all times, and thus accidents are rare (we’ve had one bike-share fatality in London, for example, and that was a commuter, not a tourist). This attitude then spills over to other road users, in addition to the gains from higher numbers of cyclists overall.

      I make a similar argument for not wearing a helmet – in my experience motorists treat me very differently with and without.

      • John Mashey says:

        Helmets: this advice seems akin to an old book:
        The Boston Driver’s Handbook: Wild in the Streets

        One piece if advice was to convince all around that you were crazy or at least not paying attention, as by simulating bird watching when entering a traffic circle. I think there’s a newer edition, but it appeared to have good advice, especially for those new to the area.

        • Phil says:

          You guys did read the blog post, right? It’s not about whether helmets are or aren’t a good idea. It’s about the ethics of publishing a false claim and then not correcting it when it’s pointed out to you.

          • zbicyclist says:

            Phil, thanks for a good blog post.

            I think it’s impossible to make a post with the words “bicycle helmet” in it without tapping into the whole decades of the helmet wars.

          • John Mashey says:

            Sure. I take it for granted that people ought to admit to error.
            I’ve spent enough time trying to get certain people to do that.

            But this struck me as one of those cases where there was actually a space for useful and interesting studies, if done properly with attention to the various conflating factors, but that such might get derailed by the mistakes in the original papers. (I.e., go back to my first comment, not the short one above.) I’m not into helmet wars (my wife and I wear them, as in skiing), but I was amused by the idea that drivers recognizing likely-to-be-bad cyclists might be a factor to be considered.

  5. […] Phil Price writes (at Andrew Gelman’s blog) on the impact of bike-share programs: […]

  6. Adam says:

    This pisses me off, and I think there’s blame to be had by the authors, the University press office, and the WaPo journalist who trumpeted the claims.

    That being said, here’s a weak argument in the other direction: If jrkrideau is right, and these guys have a pro-helmet agenda, then their argument can go like this:

    Sure, adding a bike share program reduces injuries overall, including head injuries. But it reduces head injuries less than other types of injuries (hence the proportion of all injuries that are head injuries increases). Why might this be? Well, all these new cyclists who are using the bike share program don’t use helmets, so if they’re gonna get injured, it’s more likely gonna be to the head. That is, there are two effects: the bike share program lessens injuries overall, but also increases head injuries, because a smaller proportion of bikers wears helmets. Even though the number of head injuries decreases, this 2nd effect keeps it from decreasing as much as it should. In other words, the protective effect of the bike-share program isn’t as strong as it could be, because bike-share users don’t wear helmets.

    I still don’t buy it at all, and I still think the way it’s presented is profoundly misleading (lying with statistics) but still, maybe MAYBE there’s something hidden there.

  7. If they are a good plumber then they will have the experience needed to fix it right.
    This is bad news for workers in the building trades, many of whom considered
    themselves recession proof, secure in the belief that there would always be a market for their work, because people
    would always need basic housing-related work done, such
    as roof repair or replacement plumbing. Students who attend city
    colleges often do so to acquire an associate’s degree and many tend to pursue
    state colleges or universities to earn a bachelor’s degree.

  8. Yes! Finally someone writes about san francisco county process server.

  9. […] “As if we needed another example of lying with statistics and not issuing a correction: bike-share in…”, Phil Price, andrewgelman.com, June 17, […]

Leave a Reply