In policing (and elsewhere), regional variation in behavior can be huge, and perhaps give a clue about how to move forward.

Rajiv Sethi points to a discussion of Peter Moskos on the recent controversy over racial bias in police shootings.

Here’s Sethi:

Moskos is not arguing here that the police can do no wrong; he is arguing instead that in the aggregate, whites and blacks are about equally likely to be victims of bad shootings. . . .

Moskos offers another, quite different reason why bias in individual incidents might not be detected in aggregate data: large regional variations in the use of lethal force.

To see the argument, consider a simple example of two cities that I’ll call Eastville and Westchester. In each of the cities there are 500 police-citizen encounters annually, but the racial composition differs: 40% of Eastville encounters and 20% of Westchester encounters involve blacks. There are also large regional differences in the use of lethal force: in Eastville 1% of encounters result in a police killing while the corresponding percentage in Westchester is 5%. That’s a total of 30 killings, 5 in one city and 25 in the other.

Now suppose that there is racial bias in police use of lethal force in both cities. In Eastville, 60% of those killed are black (instead of the 40% we would see in the absence of bias). And in Westchester the corresponding proportion is 24% (instead of the no-bias benchmark of 20%). Then we would see 3 blacks killed in one city and 6 in the other. That’s a total of 9 black victims out of 30. The black share of those killed is 30%, which is precisely the black share of total encounters. Looking at the aggregate data, we see no bias. And yet, by construction, the rate of killing per encounter reflects bias in both cities.

This is just a simple example to make a logical point. Does it have empirical relevance? Are regional variations in killings large enough to have such an effect? Here is Moskos again:

Last year in California, police shot and killed 188 people. That’s a rate of 4.8 per million. New York, Michigan, and Pennsylvania collectively have 3.4 million more people than California (and 3.85 million more African Americans). In these three states, police shot and killed… 53 people. That’s a rate of 1.2 per million. That’s a big difference.

Were police in California able to lower their rate of lethal force to the level of New York, Michigan, and Pennsylvania… 139 fewer people would be killed by police. And this is just in California… If we could bring the national rate of people shot and killed by police (3 per million) down to the level found in, say, New York City… we’d reduce the total number of people killed by police 77 percent, from 990 to 231!

This is a staggeringly large effect.

Additional evidence for large regional variations comes from a recent report by the Center for Policing Equity. The analysis there is based on data provided voluntarily by a dozen (unnamed) departments. Take a close look at Table 6 in that document, which reports use of force rates per thousand arrests. The medians for lethal force are 0.29 and 0.18 for blacks and whites respectively, but the largest recorded rates are much higher: 1.35 for blacks and 3.91 for whites. There is at least one law enforcement agency that is killing whites at a rate more than 20 times greater than that of the median agency.

On the reasons for these disparities, one can only speculate:

I really don’t know what some departments and states are doing right and others wrong. But it’s hard for me to believe that the residents of California are so much more violent and threatening to cops than the good people of New York or Pennsylvania. I suspect lower rates of lethal force has a lot to do with recruitment, training, verbal skills, deescalation techniques, not policing alone, and more restrictive gun laws.

This is all important in its own right but I also wanted to highlight it as an example of a more general principle about different levels of variation when considering policy interventions.

One of my favorite examples here is smoking: it’s really hard to have an individual-level intervention to help people quit smoking. But aggregate interventions, such as banning indoor smoking, seem to work. This seems a bit paradoxical: after all, aggregate changes are nothing but aggregations of individual changes, so how could it be easier to change the smoking behavior of many thousands of people, than to change behaviors one at a time? But that’s how it is. Individual decisions are not so individual, as is most obvious, perhaps, in the variation across populations and across eras in family size: nowadays, it’s trendy in the U.S. to have 3 kids; a couple decades back, 2 was the standard; and a few decades earlier, 4-child families were common. We make our individual choices based on what other people are doing. And, again, it’s really hard to quit smoking, which can make it seem like smoking is as inevitable as death or taxes, but smoking rates vary a lot by country, and by state within this country.

To return to the policing example, we’ve had lots of discussion about whether or not particular cops or particular police departments are racially biased—lots of comparisons within cities—but Moskos argues we have not been thinking hard enough about comparisons between cities. An interesting point, and it would be good to see it on the agenda.

27 thoughts on “In policing (and elsewhere), regional variation in behavior can be huge, and perhaps give a clue about how to move forward.

  1. I looked into robbery vs murder rates a while ago, to see whether there seemed to be a relationship. Using FBI reports, over the period 1985-2010 (the most uniform coverage I could find), I found that for large and medium size cities I looked at, there was pretty nearly a linear relationship with surprisingly small dispersion (that is, as viewed on a log-log plot). The actual rates varied more than an order of magnitude. Remember that this period covers a large drop in crime rates from a peak near 1985 or 1990 to the present.

    Papers I found on the number of murders committed during robberies had those rates far too low to explain the relationship. Although large cities like New York or Chicago did tend to have higher rates (of both murder and robbery), there was not clean or obvious division into large vs smaller cities. And they all were clearly on the same regression line (by eye on a log-log plot).

    OTOH, plotting the same kind of data for rural states – states like Montana, South Dakota, or Maine, where there is no large city to distort the results, the relationship is entirely different. By eye, it appears flat – robbery rates don’t increase with the murder rates – with far lower robbery rates, though the murder rates overlap the murder rates of the cities.

    There is clearly a big difference between rural states and cities in their murder vs robbery behavior. It would not be a surprise if there were big differences in policing behavior too.

  2. “Individual decisions are not so individual, as is most obvious, perhaps, in the variation across populations and across eras in family size: nowadays, it’s trendy in the U.S. to have 3 kids; a couple decades back, 2 was the standard; and a few decades earlier, 4-child families were common. We make our individual choices based on what other people are doing.”

    Is there evidence that people emulating each other is a better explanation than common external forces (e.g. the economy) for the existence of a number of children mode?

    • Z:

      I assume it’s a combination of economic factors, demographic factors, and people emulating each other. But I’ve not seen any research on the topic, so this is just my general impression.

      Maybe some expert is reading deep enough into this thread to notice this discussion and can add some information?

      • Not an expert, but the “demographic transition”, from relatively uniform high total fertility rates a couple of centuries ago to the low total fertility rates seen in most countries today has been the subject of countless studies for more than century (with no consensus in sight):

        https://scholar.google.com/scholar?hl=en&q=demographic+transition

        There is widespread agreement that this transition occurred roughly around the time that child mortality rates dropped dramatically, but my impression is that the causal role of reduced child mortality is still debated.

        In fact, there is now widespread concern that fertility rates are dropping below replacement (2) in many countries, the so-called “second demographic transition.”

        Copying others is one idea out there. This paper references some of that stuff:

        http://www.pnas.org/content/107/Supplement_2/8985.full

    • Raghuveer:

      Good point. It appears that having 3 kids is no more or less popular in the U.S. than it was a decade ago. The difference is in who is having the larger families.

      Here’s what’s up. I googled *percentage of americans with 3 kids over time*, and the first link was Family size in America: Are large families back?:

      With celebrities breaking the two-kid barrier, big families suddenly seem as trendy as jumbo-sized sunglasses and handbags.

      And if you read the news reports, you might think it’s not just Hollywood but regular American families that are going super-sized. “Get ready for the new baby boom,” proclaims a recent headline from Life magazine. “For more parents, three kids are a charm,” says USA Today. We decided to cut through the buzz and find out whether big families really are on the upswing . . .

      Here’s what they found:

      Not really, says Steve Martin, a sociologist at the University of Maryland. When Martin crunched the numbers from a 2004 government survey — the most recent available — he found that 28 percent of women age 35 to 44, who are winding up their childbearing years, have three kids or more. Ten years ago, it was 29 percent. The numbers for younger women haven’t budged much, either. . . .

      So why does it suddenly seem like you can’t walk down the street without tripping over a double-wide stroller and a few toddlers?

      Despite the nationwide numbers, big broods could be a trend in certain areas, says S. Philip Morgan, sociologist and demographer at Duke University. “You do get clusters of behavior that are very real,” he says. “But it’s not appropriate to generalize them across the country, because there are other pockets that are behaving very differently.”

      Also:

      While the percentage of moms having Brady Bunch-sized broods has held steady, the women who make up their ranks have changed somewhat. . . . Professional moms have twice as many kids at home, on average, than their high-powered counterparts did back in 1977, according to a 2002 report from the Families and Work Institute. And in a 2000 study, sociologist Martin found that college-educated women who put off motherhood until their 30s are suddenly having families almost as big as everyone else’s. “That’s historically unprecedented,” he says.

      And:

      Wealthier families in general seem to be warming up to the idea of moving past a tasteful two. “Our survey from 2002 found that 12 percent of higher-income women had three or more children,” says Anjani Chandra, a researcher at the National Center for Health Statistics. “The figure from 1995 is only about 3 percent.”

      This is all consistent with Z’s comment above regarding economic factors, in that the increase in #kids for rich families is coinciding with an increase in income and wealth among richer people. Still, an increase from 3% to 12% is a big deal no matter how you slice it.

      In summary, I made the classic pundit error of generalizing from my social class to “the U.S.” as a whole.

      • Thanks, especially since I realize that this is a tangent from your actual post.

        The quote “You do get clusters of behavior that are very real” does raise red flags of chasing noise, though! (Randomness routinely leads to clusters that look “very real,” since people have a terrible intuition for randomness.) Agreed on the 3-12% shift being large, though, assuming the data are correct.

        On your last line of “generalizing from my social class” — I will now update my mental picture of you as having “jumbo-sized sunglasses and handbags.”

  3. The smoking analogy, is that really about individual vs aggregate? Seems more like a nudge vs a hard-law difference.

    e.g. If you passed draconian laws targeting specific smokers trying to quit where they wore a sensor-bracelet that gave them a $1000 fine or a night in jail each time they smoked it could be an individual-level intervention that works?

  4. re: “within” and “between” comparisons:

    I find one of the most annoying aspects of observational social science research to be the reliance on a single “preferred” point estimate, one that often relies exclusively on either “within” or “between” variation, or occasionally some type of “difference in difference” that utilizes both. But it seems to me a mistake that we don’t try to learn from all the sources of variation whenever they are available.

    I just think this within/between comparisons distinction is a really powerful conceptual tool that is fundamentally intuitive, easy to implement, and interpretable to lay-people. So maybe that’s why academics don’t love it – it’s too easy! But I would much prefer that when people have some sort of repeated cross-sections or panel data, and they use “time” and/or “individual/group” fixed-effects, they would stop writing “this controls for…” and start writing “this forces comparisons…”

    And now I guess you could chime in that multi-level Bayesian modeling does a great job at trading-off identifying from within/between group variation, and all you need is 7 schools. But I want to see the pure-within and pure-between estimates too.

    • > why academics don’t love it – it’s too easy!
      I think it more that academics what to be tough guys and figure out the one best (or few best) way to definitively analyse the data.

      As an example, in an earlier draft of this paper http://cid.oxfordjournals.org/content/28/4/800.full.pdf, I was suggesting numerous non-definitive analyses and the tough guy statistical group from the south insisted they could find the best model and only include that in the paper. I was asked to drop out as an author and a draft was submitted. Fortunately the journal reviewers were so negative that I was asked to rejoin and a paper that more reasonably reflected the insurmountable challenges of analyzing an observation intervention study resulted.

      Remember discussing the experience with some more experienced statisticians at a JSM and indicating getting this into the paper:
      “Because of uncertainty about optimal model selection given the small sample size, we looked at the estimated IVIG effect for all possible models. The minimum estimated odds ratio for survival associated with an IVIG effect was 4.3.”

      One was very surprised I _got away with that_ and asked that I mail them a copy.

      • First – note to self: don’t get streptococcal toxic shock syndrome.

        Second – re: getting away with stuff…. Here’s hoping I get away with the 50 or so p-values I provide for a particular test in my new paper. It is a new test, and no one has experience with exactly how to specify it, so I specify it a bunch of ways and show all the results in a graph (along with the exact same specifications of a placebo test where I know it should fail to reject). Seems much less like “getting away with something” than just showing the one p-value most conducive to whichever interpretation I prefer.

        Third – I get your point about “tough guy” statistics. But can’t a tough guy also recognize that different “models” are actually identified using different comparisons and that the differences in the point estimates are telling us important information about who is affected and how much they respond? I mean, once you accept that different “models” are actually estimating different aspects of the world (and not all trying to get at the same aspect of the world, the same “parameter”) then there is no reason to look for the “right” analysis anymore, it is about using all the information to make interesting, nuanced inferences from your different analyses. I guess my point is just that the motivation to be a “tough guy” who gets the “right” or “best” answer is prefaced on the false ideas that there a) is one answer and b) that all models estimating some coefficient are actually estimating the same parameter. Maybe this is more important with a concept like “Labor Supply” than with “Ability of Drug to Keep You Alive”.

  5. Right, because how can you really know how much to weigh the two extreme cases to develop an overall estimate? Which is more or less equivalent to asking how well you can support one prior or another, absent any other information. At least if you can see the range of the kinds of estimates, you may find some plausible way to weigh one more heavily.

  6. I did some investigation into the two law enforcement killings in my Los Angeles neighborhood in this decade:

    One was a classic suicide-by-cop: a white guy sits down on Ventura Blvd’s sidewalk and starts firing into the air until LAPD kills him. In retrospect, he was clearly not trying to kill bystanders, just get himself killed. I feel bad for the cop who had to shoot him. Perhaps in the future we could develop methods for averting suicide-by-cop?

    The other police killing was a chain of screw-ups that ended with a federal agent killing an 18-year-old violist (white). Law enforcement tried to spin it as nefarious elements hanging out in the upscale parking lot, but I ran into the bereaved mother looking for clues at the scene and told her that as a long time local, the the cops’ story sounded phony and she should sue. She did and won $3 million from a judge.

    Both dead guys were white so nobody much cared about these shootings outside of friends and family.

    But both seem like the kind of system problems that research and training could make less likely. But all the energy is focused on proving that white racism is the culprit in police shootings rather than on improving police systems universally across races. So not much gets accomplished because the Obama Administration and the media want to obsess over white racism rather than law enforcement not having the ideals methods for dealing with difficult situations.

  7. I wonder if the rise of video cameras inclines cops to use less non-lethal force.

    For example, the LAPD traditionally used a chokehold to subdue violent resistors, occasionally killing somebody through asphyxiation. That was banned, so they switched to beating recalcitrants with night sticks, such as Rodney King. That looked really bad on video, especially because the shakey first few seconds of the video establishing what King did were edited out of broadcast. In the retrial of the cops, the jury eventually convicted, but three jurors told the newspapers that only the last of about 60 blows could not be justified by King’s resistance.

    Maybe post Rodney King it seemed faster and less likely to wind up on nightly news just to shoot somebody and get it over with quick than to non-lethally restrain them will billy clubs or choke holds. But now cameraphones are omnipresent, so even quick shootings are recorded.

  8. The example illustrates how individual bias can hide in aggregate stats, but then that example is applied to actual data. The regional differences can also be explained by regional crime differences which is not discussed. You need a measure of excess deaths. This can be a threshold design, i.e. once an individual acts such that lethal force is justified, does a white individual linger in that zone longer or have to do something more outrageous to be killed or etc.? Excess deaths are then relative to the lower threshold. Deaths per police counter is an entirely inadequate measure for such a complex decision.

    Really you would need to code the circumstances of each death according to whether it was justified. You would then need to see whether there are regional differences in justified police lethal force. Then we can argue about about changing those rules. Of course if it is unjustified, there’s a cover up, the reports are fabricated, etc then that is already punishable. Perhaps the regional variation can suggest deeper inquiry but they are not dispositive. As I recall Fryer could only attempt such a coding in Houston, which says better measurement is sorely needed.

  9. The problem with the story is that the aggregate bias is substantially in the other direction. It would take pretty wild patterns of bias and differential lethal force use to save anti black bias from the aggregate data. Blacks and whites are arrested roughly equally for violent crimes ( in terms of absolute numbers), yet, judging by Guardian numbers, blacks are only a quarter of victims of police shootings, while whites constitute 50% of the sample. Again, there is no evidence that composition effects can come even close to explaining this pattern.

    The only hope Rajiv and his ilk have is to argue that the arrest data are biased. In other words substantial numbers of blacks are being railroaded by the cops to skew the numbers. Obviously, this would require pretty massive conspiracy. Even worse, you also have the homicide data, which are a tad hard to fake. They show that roughly 50% of homicide victims are black. Given that violent crime is known to correlate well with homicide, it’s very hard to believe that the arrest data are in any way substantially skewed. Unless, you imagine, that posses of KKK, unbeknownst to anybody, are riding inner cities in search for deadly entertainment.

      • Yes, the numbers are interesting, but the high variability is not likely to eliminate (the negative) aggregate bias. The US has a problem with the levels of fatal police violence. However, there is little evidence so far that we have a problem with racist police.

        Rajeev’s ilk are people who jump to conclusions based on anecdotes and prejudice, and search for racism wherever they can.

  10. Moskos doesn’t really mean that there are regional variations in the sense that region is the real variable in question. He means there are variations from organization to organization, and therefore the way to approach the issue of reducing police shootings is to work at the organizational level. As he says, if everywhere would go as low as NYPD in terms of police shootings per capita the total would be much lower. So the thing is to look at how, say, NYPD does training of officers, what policies it enforces, and how the environment (e.g. lower gun availability and a much lower violent crime rate than most other big cities) combine to create this. The point being that, for example, LA could change not that somehow the weather in LA produces more police discharges.

    • I think something that shouldn’t be underestimated is in fact the weather. How many people commit crimes in the middle of a Pennsylvania sleeting/freezing-rain/blizzard? In LA every single day you can be out committing crimes.

      Also, another thing that shouldn’t be underestimated is the role of borders and drug importation. What fraction of Mexican cartel drugs comes through So Cal? It’s gotta be a LOT.

      The endemic violent gang structure in south-central LA is legendary, quite literally. It’s imbued into a whole industry of “gangsta rap” storytelling. It’s become a self-perpetuating nightmare of violence.

      In my trips to NY and Philly, I’ve also noticed a relatively strong and growing black middle class. People working at restaurants or the Port Authority or other transit areas etc where travelers are include notably large populations of multiple races. In Los Angeles, the equivalent is a large latino population, doing yard care, construction, restaurant work, etc. I don’t know that I’ve ever once seen a black person waiting a table, or doing a UPS delivery (in Pasadena, Altadena, Glendale, Burbank, etc). There are whole neighborhoods near me full of immigrant Chinese, Vietnamese, Korean, Armenian, and Latino ethnic groups. They don’t have the “no fly zone” around them. But South Central is pretty much “Stay out” unless you’re driving through on the 50 foot raised 110 freeway at 80 MPH.

      I don’t dispute that training methods and soforth could help a lot, but I also think there are just some inevitably difficult demographic, geographic, climate, and other factors that actually have an important role which can’t be transported between locations the way training can.

      • For those that aren’t from this area, LA County is one enormous urban area, with the borders between cities, or districts noticeable only because of the street sign that says you’re entering location X. Literally you could drive on surface streets in one direction averaging 25 MPH for like 5 hours and never leave an urban area (though you’d probably get out of LA county into San Bernardino or Orange, depending on which direction you decided to go)

        Because of this there’s a tendency to think of LA as the whole county of Los Angeles. But, when it comes to policing, the LAPD handles the city of LA only. The city of LA is highly variable, but for the most part, people who have any kind of wealth just stay out. In that respect it’s a little like Philly, surrounded by suburbs where the wealth lives.

      • Also, I should clarify, it’s not that I’ve never seen any successful say upper-middle class black people in this area. In fact I know professors, TV writers, a fireman, and several other black families who are quite successful. What I haven’t seen here is a path where poorer black people can get into a middle-class environment. Most of the middle and upper middle class black families I know came from the midwest, or east coast, or other parts of CA. That stuck-in-place situation seems to breed a lot of violent encounters with the police.

    • This speaks to a larger issue. The level of fatal police violence in the US is simply shockingly high, and not justified at all by the higher crime rates. There’s clearly a problem with accountability.

Leave a Reply to Tom Passin Cancel reply

Your email address will not be published. Required fields are marked *