Guy Molyneux writes:
I saw your latest post on the hot hand too late to contribute to the discussion there. While I don’t disagree with your critique of Gilovich and his reluctance to acknowledge past errors, I do think you underestimate the power of the evidence against a meaningful hot hand effect in sports. I believe the balance of evidence should create a strong presumption that the hot hand is at most a small factor in competitive sports, and therefore that people’s belief in the hot hand is reasonably considered a kind of cognitive error. Let me try to explain my thinking in a couple of steps.
I think everyone agrees that evidence of a hot hand (or “momentum”) is extremely hard to find in actual game data. Across a wide range of sports, players’ outcomes are just slightly more “streaky” than we’d expect from random chance, implying that momentum is at most a weak effect (and even some of that streakiness is accounted for by player health, which is not a true “hot hand”). This body of work I think fairly places the burden of proof on the believers in a strong hot hand, or those still open to the idea (like you), to show why this evidence shouldn’t end the debate. Broadly speaking, two serious objections have been raised to accepting the empirical evidence from actual games.
First, you argue that “Whether you made or missed the last couple of shots is itself a very noisy measure of your ‘hotness,’ so estimates of the hot hand based on these correlations are themselves strongly attenuated toward zero.” If most of the allegedly hot players we study were really just lucky, and thus quickly regress to their true mean, the elevated performance by the subset of truly ‘hot’ players will be masked in the data. I take your point. Nonetheless, given the absence of observed momentum in games, one of two things must still be true: A) the hot hand effect is large but rare (your hypothesis), or B) the hot hand effect is small but perhaps frequent. This may be an important distinction for some analytic purpose, but from my perspective the two possibilities are effectively the same thing: the game impact (I won’t say ‘effect’) of the hot hand is quite small. By “small” I mean both that the hot hand likely has a negligible impact on game outcomes, and that teams and athletes should largely ignore the hot hand in making strategic decisions.
And since the actual impact on games is quite small *even if* your hypothesis is correct (because true hotness is rare), it follows that belief in a strong hot hand by players or fans still represents a kind of cognitive failure. The hot hand fallacy held by most fans, at least in my experience, is not that a very few (and unknowable) players sometimes get very hot, but rather that nearly all athletes sometimes get hot, and we can see this from their performance on the field/court.
(An important caveat: IF it proved possible to identify “true” hot hands in real time, or even to identify specific athletes who consistently exhibit true hot hand behavior, then my argument fails and the hot hand might have legitimate strategic implications. But I have not seen evidence that anyone knows how to do this in any sport.)
The second major objection made to the empirical studies is that the hot hand is disguised as a result of player’s very knowledge of it. As Miller and Sanjurjo suggest, “the myriad confounds present in games actually make it impossible to identify or rule out the existence of a hot hand effect with in-game shooting data, despite the rich nature of modern data sets.” Two main confounds are usually cited: hot players will take more difficult shots, and opposing athletes will deploy additional resources to combat the hot player. Some have argued (including Miller and Sanjurjo) that these factors are so strong that we must ignore real game data in favor of experimental data. But I think it is a mistake to dismiss the game data, for three reasons:
- The theoretical possibility that players’ shot selections and defensive responses could perfectly – and with astonishing consistency – mask the true hot hand effect is only a possibility. Before we dismiss a large body of inconvenient studies, I’d argue that hot hand believers need to demonstrate that these confounds regularly operate at the necessary scale, not just assume it.
- A sophisticated effort to control for shot selection and defensive attention to hot basketball shooters concludes that the remaining hot hand effect is quite modest. Conversely, as far as I know no one has shown empirically that the enhanced confidence of hot players and/or opponents’ defensive responses can account for the lack of observed momentum in a sport.
- Efforts to detect a hot hand effect in baseball have invariably failed. And that’s important, because in baseball the players cannot choose to take on more arduous tasks when they feel “hot,” and opposing players have virtually no ability to redistribute defensive resources in a way that disadvantages players perceived to be hot. So even if you reject the Sloan study and think confounds explain the lack of momentum in basketball, they cannot explain what we observe in baseball.
I would also note that this “confounds” objection is in fact a strong argument *in favor* of the notion that the hot hand is a cognitive failure, given your argument that in-game streaks are a very poor marker of true hotness. If the latter is true, then it would still be a cognitive error for a player or his opponents to act on this usually-false indicator of enhanced short-term talent. If players on a streak take more difficult shots, they are wrong to do so, and teams that change defensive alignments in response are also making a mistake.
So, these are the reasons I remain unpersuaded that I should believe in a hot hand in the wild, or even consider it an open question. That leaves us, finally, with the experimental data that some feel should be privileged as evidence. I haven’t read enough of the experimental research to form any view on its quality or validity. But for answering the question of whether belief in the hot hand is a fallacy, I don’t see how the results of these experiments much matter. Fans and athletes believe they see the hot hand in real games. If a pitcher has retired the last nine batters he faced, many fans (and managers!) believe he is more likely than usual to get the next batter out. If a batter has 10 hits in his last 20 at bats, fans believe he is “on a tear” and more likely to be successful in his 21st at bat (and his manager is more likely to keep him in the lineup). But we know these beliefs are wrong.
Even if experiments do demonstrate real momentum for some repetitive athletic tasks in controlled settings, this would not challenge either of my contentions: that the hot hand has a negligible impact on competitive sports outcomes, and fans’ belief in the hot hand (in real games) is a cognitive error. Personally, I find it easy to believe that humans may get into (and out of) a rhythm for some extremely repetitive tasks – like shooting a large number of 3-point baskets. Perhaps this kind of “muscle memory” momentum exists, and is revealed in controlled experiments. But it seems to me that those conducting such studies have ranged far from the original topic of a hot hand in competitive sports — indeed, I’m not sure it is even in sight.
I don’t know that I have anything new to say after a few zillion exchanges in blog comments, but I wanted to put Guy’s reasoning out there, because (a) he expresses it well, and (b) he’s arguing that I’m a bit off in my interpretation of the data, and that’s something I should share with you.
The only thing I will comment on in Guy’s above post is that I do think baseball is different, because a hitter can face different pitches every time he comes to the plate. So it’s not quite like basketball where the task is the same every time.
P.S. Yeah, yeah, I know, it seems at times that this blog is on an endless loop of power pose, pizzagate, and the hot hand. Really, though, we do talk about other things! See here, for example. Or here. Or here, here, here.
P.P.S. Josh (coauthor with Sanjurjo of those hot hand papers) responds in the comments. Lots of good discussion here.