John Donohue sent me this paper reviewing evidence about the deterrent effect of the death penalty. I recommend the Donohue and Wolfers paper highly–at least on the technical side, it’s the paper I would like to have written on this topic, if I had been able to go through all the details on the many recent papers on the topic. I can’t really comment on their policy recommendations but have some futher thoughts on the death-penalty studies from a statistical perspective.
I’ll first give the key paragraph of the conclusion of their paper, then give my comments:
We [Donohue and Wolfers] have surveyed data on the time series of executions and homicides in the United States, compared the United States with Canada, compared non-death penalty states with executing states, analyzed the effects of the judicial experiments provided by the Furman and Gregg decisions comparing affected states with unaffected states, surveyed the state panel data since 1934, assessed a range of instrumental variables approaches, and analyzed two recent state-specific execution moratoria. None of these approaches suggested that the death penalty has large effects on the murder rate. Year-to-year movements in homicide rates are large, and the effects of even major changes in execution policy are barely detectable. Inferences of substantial deterrent effects made by authors examining specific samples appear not to be robust in larger samples; inferences based on specific functional forms appear not to be robust to alternative functional forms; inferences made without reference to a comparison group appear only to reflect broader societal trends and do not hold up when compared with appropriate control groups; inferences based on specific sets of controls turn out not to be robust to alternative sets of controls; and inferences of robust effects based on either faulty instruments or underestimated standard errors are also found wanting.
My first comment is that death-penalty deterrence is a difficult topic to study. The treatment is observational, the data and the effect itself are aggregate, and changes in death-penalty policies are associated with other policy changes. (In contrast, my work with Jim Liebman and others on death-penalty reversals was much cleaner–our analysis was descriptive rather than causal, and the problem was much more clearly defined. This does not make our study any better, but it certainly made it easier.) Much of the discussion of the deterrence studies reminds me of a little-known statistical principle, which is that statisticians (or, more generally, data analysts) look best when they are studying large, clear effects. This is a messy problem, and nobody is going to come out of it looking so great.
My second comment is that a quick analysis of the data, at least since 1960, will find that homicide rates went up when the death penalty went away, and then homicide rates declined when the death penalty was re-instituted (see Figure 1 of the Donohue and Wolfers paper), and similar patterns have happened within states. So it’s not a surprise that regression analyses have found a deterrent effect. But, as noted, the difficulties arise because of the observational nature of the treatment, and the fact that other policies are changed along with the death penalty. There are also various technical issues that arise, which Donohue and Wolfers discussed.
Also some quick comments:
– Figure 1 should be two graphs, one for homicide rate and once for execution rate.
– Figure 2 should be 2 graphs, one on top of the other (so time line is clear), one for the U.S. and one for Canada. Actually, once Canada is in the picture, maybe consider breaking U.S. into regions, since the northern U.S. is perhaps a more reasonable comparison to Canada.
– Figure 8 qould be much clearer as 12 separate time series in a grid (with common x- and y-axes). It’s hopeless to try to read all these lines. With care, you could fit these 12 little plots in the exact same space as this single graph.
Here’s an article by Joanna Shepherd stating the position that the death penalty deters homicides in some states but not others. Their story is possible but it seems to me (basically, for the reasons discussed in the Donohue and Wolfers paper) not a credible extrapolation from the data.
The death penalty as a decision-analysis problem?
Policy questions about the death penalty have sometimes been expressed in terms of the number of lives lost or saved by a given sentencing policy. But I think this direction of thinking might be a dead end. First off, as noted above, it may very well be essentially impossible to statistically estimate the net deterrent effect of death sentencing–what seem like the “hard numbers” (in Richard Posner’s words, “careful econometric analysis”) aren’t so clear at all.
More generally, though, I’m not sure how you balance out the chance of deterring murders with the chance of executing an innocent person. What if each death sentence deterred 0.1 murder, and 5% of people executed were actually innocent? That’s still a 2:1 ratio (assuming that it’s OK to execute the guilty people). Then again, maybe these innocent people who were executed weren’t so innocent after all. But then again, not every murder victim is innocent either. Conversely, suppose that executing an innocent person were to deter 2 murders (or, conversely, that freeing an innocently-convicted man were to un-deter 2 murders). Then the utility calculus would suggest that it’s actually OK to do it. In general I’m a big fan of probabilistic cost-benefit analyses (see, for example, chapter 22 of my book), but here I don’t see it working out. The main concerns–on the one hand, worry about out-of-control crime, and on the other hand, worry about executing innocents–just seem difficult to put on the same scale.
Finally, regarding decision analysis, incentives, and so forth: much of the discussion (not in the Donohoe and Wolfers paper, but elsewhere) seems to go to the incentives of potential murderers. But the death penalty also affects the incentives of judges, juries, prosecutors, and so forth. One of the arguments in favor of the death penalty is that it sends a message that the justice system is serious about prosecuting murders. This message is sent to the population at large, I think, not just to deter potential murderers but to make clear that the system works. Conversely, one argument against the death penalty is that it motivates prosecutors to go after innocent people, and to hide or deny exculpatory evidence. Lots of incentives out there.