How can and should we interpret regression models of basketball?

I read Malcolm Gladwell’s article in the New Yorker about the book, “The Wages of Wins,” by David J. Berri, Martin B. Schmidt, and Stacey L. Brook. Here’s Gladwell:

Weighing the relative value of fouls, rebounds, shots taken, turnovers, and the like, they’ve created an algorithm that, they argue, comes closer than any previous statistical measure to capturing the true value of a basketball player. The algorithm yields what they call a Win Score, because it expresses a player’s worth as the number of wins that his contributions bring to his team. . . .

In one clever piece of research, they analyze the relationship between the statistics of rookies and the number of votes they receive in the All-Rookie Team balloting. If a rookie increases his scoring by ten per cent—regardless of how efficiently he scores those points—the number of votes he’ll get will increase by twenty-three per cent. If he increases his rebounds by ten per cent, the number of votes he’ll get will increase by six per cent. . . . Every other factor, like turnovers, steals, assists, blocked shots, and personal fouls—factors that can have a significant influence on the outcome of a game—seemed to bear no statistical relationship to judgments of merit at all. Basketball’s decision-makers, it seems, are simply irrational.

I have a few questions about this, which I’m hoping that Berri et al. can help out with. (A quick search found that this blog that they are maintaining.) I should also take a look at their book, but first some questions:

1. Reading Gladwell’s article, I assume that Berri et al. are doing regression analysis, i.e., estimating player abilities as a linear combination of individual statistics. I have the same question that Bill James asked in the context of baseball statistics: why restrict to linear functions? A function of the form A*B/C (that’s what James used in his runs created formula, or more fully, something like (A1 + A2 +…)*(B1 + B2 +…)/C) could make more sense.

2. Have Berri et al. looked at the plus-minus statistic, which is “the difference in how the team plays with the player on court versus performance with the player off court”? (See here for some references to this, also here and here.) When I started reading Gladwell’s article, I thought he was going to talk about the plus-minus statistic, actually.

3. I’m concerned about Gladwell’s causal interpretation of regression coefficients. I don’t know what was in the analysis of all-star voting, but if you run a regression including points scored and also rebounds, turnovers, etc., then the coefficient for “points scored” is implicitly comparing two players with different points scored but identical numbers of rebounds, assists, etc.–i.e., “holding all else constant.” But that is not the same as answering the what happens “if a rookie increases his scoring by ten per cent.” If a rookie increases his scoring by 10%, I’d guess he’d get more playing time (maybe I’m wrong on this, I’m just guessing here), thus more opportunities for rebounds, steals, etc.

Just to be clear here: I’m not knocking the descriptive regression. In particular, you can play with it to model what might happen if players are switched in an out of teams (as long as you think carefully about issues such as playing time, I suppose). I’m just sensitive to mistakenly-causal interpretations of regression coefficients–the idea that you can change one variable while holding all else constant.

4. Gladwell’s article is subtitled, “When it comes to athletic prowess, don’t believe your eyes,” and he writes, “We see Allen Iverson, over and over again, charge toward the basket, twisting and turning and writhing through a thicket of arms and legs of much taller and heavier men—and all we learn is to appreciate twisting and turning and writhing. We become dance critics, blind to Iverson’s dismal shooting percentage and his excessive turnovers, blind to the reality that the Philadelphia 76ers would be better off without him.” But it seems here that the problem is not that people are igoring the statistics, but that they’re using the wrong (or overly simplified) statistics. After all, he points out in the first paragraph of his article that Iverson has led the league in scoring and steals, and his team has done well. Even if he didn’t look cool flying to the basket, Iverson might have gotten recognition from these statistics, right? This is a point that Bill James made (with regard to batting average in Fenway Park, ERA in Dodger Stadium, etc.): people can overinterpret statistics in isolation.

5 thoughts on “How can and should we interpret regression models of basketball?

  1. Andrew, with regard to your item (2): I like the plus-minus statistic — how does the team perform with Player A on the court, compared to how they perform without him — but as some of the other articles you point to probably say, this measure can be problematic. A major issue is efficiency: there can be a lot of "noise" — stochastic variability — in the stat. Fortunately (from the point of view of a plus-minus statistic) basketball has a lot of scoring and a long season, so stochastic variability is much less of a problem than in, say, baseball or hockey. But still, you really want/need to adjust for quality of the other players on the court (on both teams). It's also desirable to adjust for the game situation: in a blowout victory or loss, little or nothing is learned from performance in "garbage time," for example. Of course, some of the same problems come up with any analysis, whether plus-minus or not. No matter how you slice it, you're looking at a non-trivial bit of analysis. That said, I like the plus-minus approach.

    As for your complaint about causal interpretation, I think you're correct that Gladwell's writing is a bit confused in this section and it's not quite clear _exactly_ what he is asserting — but I also think you're being a bit pedantic. What Gladwell means is that if you compare two rookies, and they have similar rebounds but one of them scores 10% more points than the other, then the higher-scoring one will get, on average, about 20% more All-Rookie-Team votes than the lower-scoring one, no matter how they do with turnovers, steals, assists, or fouls. We can always hope for more precision in writing, but in this case I think complaining about the imprecision is a quibble: the basic point is clear enough.

  2. Anon – Andrew's point is not a quibble. It is a fundamental misunderstanding of multiple regression to think in terms of "holding all else constant". Turnovers, steals, assists, fouls, etc. are going to be correlated with rebounds to varying degrees and in real life, when one of those vary, most of the others will too.

  3. Huh, don't know why my previous comment was listed as "anonymous", I thought I had signed in.

    Kaiser (and Andrew), I understand Andrew's point, and I agree with it (obviously, since it is plainly true). I'm not saying it's not true. Sure, if a rookie were to increase his scoring by 10% and change nothing about his game, maybe his all-rookie-team voting would increase by 26% rather than 23% (on average) because he would get more playing time and thus garner more rebounds. It would have been nice for Gladwell to point out this effect, perhaps even as a cautionary tale for people trying to interpret statistical results. But I still say it's a quibble. Getting it right would have been nice, but it wouldn't change the main points of his article at all.

    I guess it's worth noting that a rookie _could_, in principle, increase his scoring by 10% and thus get more playing time, without affecting the other stats. Maybe he works harder to get points, but works less hard to collect steals and rebounds, for example. "Now you're just quibbling," I hear you say. "Exactly," I reply.

  4. this is kind of off topic from basketball and regression, but i thought it might be related enough to be interesting.

    i just put online a markov chain monte carlo estimation tool for the 2006 world cup of soccer. might be fun to experiment with (it runs right in the browser).

    link here

    i would have sent you the link via email but can't find a place to send it..

  5. [Iverson has led the league in scoring and steals, and his team has done well]

    but yet he isn't a good player, because his "shooting percentage is lousy"? Who are you gonna believe, me or your lying eyes?

    For some reason, this kind of statistics nerd got chased out of cricket a while ago (we still have statistics obsessives, but the monovariablities which appears to afflict a lot of sabermetricians isn't such a problem). The thing that really did it for them is that there are quite a lot of cricket players who don't play well a lot of the time, but who can be relied upon to raise their game hugely when required.

    Boxing provides the reductio ad absurdum of this approach; I can just see statistics nerds standing over an unconscious body explaining that based on punch-complete percentage, work rate, etc etc, they are looking at the real champion.

Comments are closed.