Skip to content
 

Kalesan, Fagan, and Galea respond to criticism of their paper on gun laws and deaths

The other day we posted some remarks on a recent paper by Bindu Kalesan, Jeffrey Fagan, Sandro Galea, “Firearm legislation and firearm mortality in the USA: a cross-sectional, state-level study.”

In response to the criticisms from me and various commenters, the authors of the paper prepared a detailed response, which I’m linking to here.

They write:

We believe that our study improves over previous efforts by examining the simultaneous effects of specific gun laws whose implementation varies from state to state. Most prior work has looked at the cumulative effects of state laws or used sui generis metrics of “legislative strength.” We focused instead on the role played by specific gun laws, seeing this as a key to inform policy by identifying “relevant and effective legislation”. We also estimated the effects of gun law changes over a three-year period, advancing prior static analyses of state gun laws. We tried to exhaust the potential of this method to estimate the range and magnitude of legal effects. . . .

Not surprisingly, some researchers have raised valid questions about the paper. The questions focus primarily on the study design and the large estimates it produced. We address these questions below. . . .

You can read the whole thing here.

49 Comments

  1. This response strikes me as a bit clueless. The point being made is that inferring causality (ie. that the presence or absence of the law is what causes observed changes) seems implausible, and then their response is basically to ignore the difficulty with the causality issue and then proceed as if their “effect estimates” are causal estimates suggesting that widespread adoption of certain laws would reduce crime to 40% of its current value or less…

    Their figure 1 shows regression lines for “have none of the 3 laws” vs “have any of the 3 laws” and Basically the entire effect is down to the existence of Hawaii. But Hawaii is a totally different kind of place than the rest of the US. If you remove HI the remaining states that “have any of the three laws” clearly have a slope of death-rate vs firearm ownership % that is HIGHER than the “none” slope.

    Also, looking at changes between two time-points 2008 and 2010 is a serious methodology problem. Using an entire timeseries is the only way you’re going to get away from a lot of noise.

    The main causality mechanism that I would guess is at work here is that certain states attract certain populations, who then pass certain laws. The existence of the laws is itself fairly marginal in terms of altering crime, but the existence of the certain population and the things that attract them is very significant. In other words, gentrification, soaring costs of living, social norms imposed by urban living, and elite populations with political influence influences both laws and crime trends.

    • jrc says:

      I think they aren’t even using two time points. They are using outcome data from 2010 and laws from 2008-10. A panel approach (yearly outcome and law data for each state) would be the obvious way to deal with the first order concern of confounding long-term differences in outcomes across states with their decisions regarding gun laws. At least by de-meaning within states they could compare changes in gun laws to changes in outcomes, which, to me, would be a sizable improvement.

    • Stephen says:

      It’s certainly an outlier but I think you are exaggerating the influence of Hawaii (Fig1).

      • The dashed horizontal line is supposed to be the regression through the triangular gray points. If you removed the HI point, I think the regression line would look similar to a line through Connecticut and Illinois, or Pennsylvania and New Jersey

        (look for the points Pennsylvania, Maryland, and California hidden in the rest of the point cloud)

        Massachusetts is also a bit of an outlier, but even it can’t really pull that line down flat like that considering that it’s in the middle of the x range, the main thing it can do is move the offset, not the slope.

        • Elin says:

          Their point is that the metric of just counting laws is wrong, it’s the kind of law that matters. However, if they took Hawaii out there would certainly be, as you seem to say, a relationship between gun ownership and firearms mortality rate even among the states with these laws. So correct, take away Hawaii and it could be that it all still comes down to reducing the ownership rate, whether through those laws or because of culture (which is the self selection issues, i.e. low gun ownership states also are states where these gun regulations are not just politically possible but also have support). So the states with these laws have lower ownership and lower mortality. Is it legal environment -> ownership -> mortality, ownership -> legal environment->mortality, [ownership+legal environment] ->mortality. Or, of course, something more complex (most likely of all).

          • Stephen says:

            Take away Hawaii and the slope is probably slightly steeper but there will obviously still be a difference in the intercept… i.e. there is a positive relation between gun ownership and mortality +/- 3 laws but a step change drop in mortality /+3 laws. That sounds plausible (why would +laws change the slope?) though I don’t know how it would effect their mortality predictions if laws were applied in all states.

            It would be great of course if the range of gun-ownership was similar for both groups – but the world is as the world is- Hawaii included. Hawaii which is not an outlier as such – not a measurement error, but actually, really part of the US…

            I guess I’m kind of agnostic about this study. It’s obviously a complex multiregression model of a snapshot timeframe. It’s imperfect and there are oddities I don’t quite understand. But I’m fine with that so long as the methodology is clear plus the problems and assumptions are not hidden.

            • Mikey says:

              (why would +laws change the slope?)- I don’t know, but given that the effects of the laws are modeled with an assumption that their effects are on Relative Risk rather than Absolute Risk, then a change to slope and not to intercept is what you would expect to see if the model is correct.

            • Martha (Smith) says:

              “Hawaii which is not an outlier as such – not a measurement error, but actually, really part of the US… “

              Something can be an outlier for various reasons — measurement error is only one possible reason. In particular, a visual or numerical outlier can sometimes be such because of some type of hon-homogeneity. In this case, HI indeed has some differences from the contiguous 48 that are relevant to the subject being studied — for example, in most states, guns can be bought readily in neighboring states (whether legally or not). But HI has no neighboring states. So it is different from other states in a way that plausibly could have an effect on the questions being studied.

              • Yes, this was one factor I was thinking of. I also think the economics of Hawaii are very different for both legal trade, and drug trade. Costs of imports are high, and illicit drugs are an import. Imports are relatively easy to control as the borders are small. The economy relies heavily on tourism, and crime is bad for tourism, there may be strong financial incentives to keep crime down… it’s just a plausibly very different place than say South Carolina.

              • Elin says:

                Great point.

    • But their estimate of gun ownership in Hawaii appears to be much too high. Kalesan et al. got their figures from a nationwide poll of 4000 people, which works out to an average of 80 people per state, probably less than that in Hawaii. Also, the poll was based on a nonrandom (largely self-selected) internet sample and had a 40% response rate.

      https://www.researchgate.net/publication/279629818_Gun_ownership_and_social_gun_culture
      “We used data from a nationally representative sample of 4000 US adults, from 50 states and District of Columbia, aged >18 years to assess gun ownership and social gun culture performed in October 2013.”

      Estimates of gun ownership are all problematic for one reason or another, but all three major types of these estimates put Hawaii as one of the least gun-owning states:

      Surveys: to my knowledge, no large surveys are recent, and of course people lie to surveyors, but:
      BRFSS Survey Results 2001 for Nationwide Firearms, with 201,881 participants, has Hawaii at 50th out of 51. http://www.schs.state.nc.us/SCHS/brfss/2001/us/firearm3.html

      Background Checks: NICS background checks per 100,000 residents (Dec. 2008 – May 2010) puts Hawaii 49th out of 51 for gun purchases.

      Suicide: Fleegler et al. use suicide ratios to put Hawaii as the state (of 50) with the lowest gun ownership rate. (“Percentage household firearm ownership was calculated by mean firearm suicides/total suicides (2007-2010) by state.”) – Fleegler, E., Lee, L., Monuteaux, M., Hemenway, D., & Mannix, R. (2013). Firearm legislation and firearm-related fatalities in the United States. JAMA Internal Medicine; 173(9):732-740. Retrieved from doi:10.1001/jamainternmed.2013.1286

  2. It’s well understood in criminology that criminals primarily get guns from black-market deals, often through straw-purchases, and that it’s relatively easy for them even in states with lots of firearms laws like CA or NY.

    So, since the laws have very little to do with the rates at which criminals acquire firearms, there is basically no plausible mechanisms by which passing these laws causally reduces firearms death, particularly when some of the laws are straw laws like the CA identification law (there has never been a SINGLE firearm on the market that meets the new ID law, so all firearms sold are ones on the list via grandfathering).

    BUT, there are LOTS Of plausible mechanisms whereby certain people are in CA, NJ, NY, MA, etc to chase high paying jobs and pay high costs of living, forcing out groups with higher rates of committing crimes, and/or creating urban growth that affects people socio-economically in multiple ways (maybe disincentivizing crime for example), and also imposing a lot of laws… whereas the people in NM, NV, TN, AR etc are totally different people, living in totally different density situations, with totally different social norms and totally different ideas about what the law should be.

    Remember also, that putting CA, NY, IL, MA, NJ, CT together accounts for around 85 Million people, which is what, 25% of the US population? Anyway those orders of magnitude… Putting some of the other big states, TX, FL, GA, etc which are “none” states into the mix, you can see they lie right on the same regression line through the “some” states once you’ve eliminated the obviously outlying HI.

    A substantive model would address questions like, how is it that in HI you can have lots of firearms and very few gun deaths whereas in South Carolina you don’t? Could it be something about the fact that you’re LIVING IN HAWAII rather than say the existence or not of certain gun laws? The assumption that the gun laws cause the effect is essentially saying that the difference between say South Carolina and Hawaii is that a random sample of South Carolinans are pretty much indistinguishable from a random sample of Hawaiians except for the fact that in Hawaii you need a few extra steps to acquire guns and ammunition???

    come on.

    • GabbyD says:

      “Could it be something about the fact that you’re LIVING IN HAWAII rather than say the existence or not of certain gun laws? “

      Why is “living in hawaii” so important, so as to preclude a comparison between hawaii and south carolina? what are the differences that may be relevant to gun deaths and ownership?

      • There are any number of reasons why living on an island in the middle of the pacific could plausibly be a lot different from living in South Carolina, the point is if you want to make a comparison between the two based entirely on the existence of one to three laws, you have to plausibly argue why they’re the SAME.

      • Obviously, something is going on in HI:

        http://www.civilbeat.com/2015/04/are-there-more-guns-than-people-in-hawaii/

        number of firearms registered each year (and they’re all registered in HI, at least legal sales) doubled from 40k to 80k between 2010 and 2013 and has been growing along an exponential curve for a decade or more.

        Given this rapid rise in firearm ownership while death rates stay extremely low, lowest in the 50 states, I’d say it’s more than a little implausible that the registration laws etc are the cause.

        According to here: http://ag.hawaii.gov/cpja/crime-in-hawaii-2012-annual/ in 2012, there were 243 violent crimes per 100k population in HI, and here http://www.disastercenter.com/crime/sccrime.htm there were 560 violent crimes per 100k in SC in the same year.

        So, basically, crime rates in hawaii overall are low, and are fairly independent in the short term from a rapid rise in firearms ownership. The population of Hawaii generally commits half the violent crime that the population of SC commits.

        In case it’s not clear, I chose South Carolina because it’s directly above Hawaii and almost right on the solid line in their graph, so “typical” for the states without the laws.

        If they remove HI the line on their graph changes dramatically.

        If they go one step further and simply restrict to the top say 15 most populated states (on the theory that the high population states are the ones making up the vast majority of the actual deaths and the rates are also maybe more stable relative to poisson type noise) those high population states are:

        CA: 39M
        TX: 27M
        FL: 20M
        NY: 20M
        IL: 13M
        PA: 13M
        OH: 12M
        GA: 10M
        NC: 10M
        MI: 10M
        NJ: 9M
        VA: 8M
        WA: 7M
        AZ: 7M
        MA: 7M
        (source: https://en.wikipedia.org/wiki/List_of_U.S._states_and_territories_by_population)

        You can see that NY, NJ, MA are kind of in their own corner of their graph, CA, IL, PA are up in the middle, and TX, FL, GA, OH, MI, WA, are similar to CA, IL, PA but without the laws.

        Without hawaii, the whole point cloud looks basically homogeneous except for a notable difference in certain new england states (NY, NJ, MA, CT, RI)

        To attribute a causal relationship to the laws, you’d have to assume that applying these laws *alone* would somehow bring everyone down to the same level of gun violence as in NY, NJ, and MA for example. I just don’t believe that is anything like true. New England is its own animal, as is Hawaii. The makeup of these states is very different from other areas of the county, as are the situations in which people live. Furthermore, none of these laws would REMOVE guns from the other states, so the gun ownership in SC for example would remain the same!

        The purchase laws change the RATE of firearm sales (at best) they do NOTHING for the firearm stock (at least say in the “short” term of the next 50 years, firearms are extremely durable).

        New York city has had a long trend downward in violence since 1990 or so: http://www.randominterestingfacts.com/wp-content/uploads/2014/03/NYC-crime-rates-by-police-commissioner-and-mayor.png

        but the gun laws in NY are unchanged in this time period as far as I understand, certainly NYC’s sullivan act was passed in 1911 !!

        My core point: they have done NOTHING that would let them understand causality between gun laws and death rates in this study and yet they assert that adding in these laws across the board would *causally* drop gun deaths by 80% ! This, even though none of these laws remove guns from households, so states like South Carolina would still have vastly larger gun ownership rates than NY.

        From Andrew’s quote from the article: “Projected federal-level implementation of universal background checks for firearm purchase could reduce national firearm mortality from 10·35 to 4·46 deaths per 100 000 people, background checks for ammunition purchase could reduce it to 1·99 per 100 000, and firearm identification to 1·81 per 100 000”

        It’s not just a little too good to be true. It’s complete fantasy. It’s not even the 25 coefficients and 50 data points issue that’s at work here, it’s total lack of any causal model that is plausible and takes into account any of the widely spatially and socially varying factors that contribute to violent crime.

        • Elin says:

          New York’s decline has been pretty whoroughly but somewhat inconclusively studied. One important aspect of it is that there has been highly focused targeting of illegal gun markets. Decline in youth population also probably plays a role, and maybe stop and frisk/disorder policing. Fagan actually was involved in some of that research.

    • Elin says:

      From my understanding, I think the impact would be more on “crimes of passion,” suicides, accidental deaths and the small number of high impact mass shootings by people with known mental illnesses. These laws have little to do with whether some drug market manager buys a gun on an illegal or legal gun market.

      • So, that’s the only plausible causal mechanism. And suicide accounts for more than half of all gun deaths. But, and this is key, a suicide is a suicide whether it’s a gun one or not. Looking across other countries with vastly different gun ownership laws and rates:

        https://commons.wikimedia.org/wiki/File:Suicide-deaths-per-100000-trend.jpg#/media/File:Suicide-deaths-per-100000-trend.jpg

        you see that suicide seems to be something that having a gun affects the method, but not the rate. Japan’s rate is DOUBLE the US rate, but hardly anyone owns firearms. New Zealand has basically the same rate as the US but it has far more restrictive gun laws than the US (everyone needs to get permission/license from the police, they actually come to your house to check that you have safe storage, ID is required for ammunition, etc etc).

        https://en.wikipedia.org/wiki/Gun_politics_in_New_Zealand

        So, as far as I can tell, we have to assume that you *might* reduce *firearm* suicides by implementing these kinds of laws (maybe) but you’re not going to make major headway on *total suicides*.

        “crimes of passion”, is a small fraction of total firearms deaths (total is 10,000/yr in US or so). Mass shootings are about 100 people / yr, domestic disputes etc might be 1000? The big driver of criminal misuse of a firearm is 18-25 year old males using illegally acquired handguns:

        https://commons.wikimedia.org/wiki/File:Homoffendersbyage.svg#/media/File:Homoffendersbyage.svg

        And you can bet your life that the massive upturn in 14-17 year olds during the 1990’s was gang/drug turf wars.

        The racial profile of victims is also telling:

        https://en.wikipedia.org/wiki/File:Homicide_victimization_by_race.jpg#/media/File:Homicide_victimization_by_race.jpg

        Black people are dying at what? 5-7 times the rate of white people, with a big correlated upturn in the 1990’s.

        This is serious tragedy, but it’s unlikely to be causally linked to any of these firearms laws. It’s linked to the WAR ON DRUGS and the socioeconomics of poor urban black children (well say 15-25 year olds)

        If you want a causally linked potential intervention, consider the Universal Basic Income and major changes to the drug laws in the US.

        The problem with this kind of study is: 1) it’s not even close to a causal study. 2) It’s written up as if it were causal. 3) It’s played into the media as if it were “proven” 4) and it drives attention away from what is really going on.

        • Although I mentioned it, I forgot to link to the bit about how gun homicides are pretty much committed with handguns:

          https://upload.wikimedia.org/wikipedia/commons/2/20/Ushomicidesbyweapon.svg

        • Corey says:

          So, as far as I can tell, we have to assume that you *might* reduce *firearm* suicides by implementing these kinds of laws (maybe) but you’re not going to make major headway on *total suicides*.

          These are static comparisons across cultures, so they’re fraught with confounders. It would be really handy if there were some natural-experiment-type situation where, within a single country, a rapid-death technology that people kept in their homes suddenly disappeared. And there is! And it does appear that the availability of a convenient, rapid, painless method of killing one’s self is indeed a direct cause of suicide.

          • Elin says:

            I think the gas change has been hugely influential in helping people see the ways in which opportunity structures really shape what people do. Make it take 15 minutes longer to commit suicide and you may reduce suicide because either people change their minds or someone can intervene or maybe something else. I just did a quick look and it does seem that suicides by gun and suicides in general went down in Australia. It will be interesting to see if the automatic braking change will result in lower suicide by car.

          • Thanks for the link. But the time frame from 1950 to mid 1970’s is also a confounder here. We see ~ 33% reduction in total suicides, over a period of say 20-30 years. Coming out of the aftermath of WWII during a period of high growth and psychological recovery from the war. Did overall suicides drop because of CO in cooking gas, or because those people who were horribly scarred by the war committed suicide between 1945 and 1955, and anyone who survived to 1955 saw a period where GDP per capita was growing between 5 and 10 % per year consistently, and the country was getting rebuilt and back on its feet?

            I think the robustness of the overall suicide rate (order of magnitude) across vast swaths of different countries is evidence that suicide is something that we don’t have strong control over. Can we reduce it 20 or 30 percent by strong efforts? Yes. Can we reduce it 50 to 90%? Probably not. Sure, I’ll take a 20 or 30% reduction, but I don’t think we can read into the coal gas change that we have that kind of control.

            • Corey says:

              Here’s another example without WWII confounding.

              However, the paper’s findings about suicide were statistically significant — and astounding. Buying back 3,500 guns correlated with a 74 percent drop in firearm suicides. Non-gun suicides didn’t increase to make up the decline.

              The buyback took place over two years: 1996-97.

              • But none of these laws involved buybacks, they involve making purchases of guns and purchase of ammunition etc more regulated.

                I think it’s pretty plausible that removing firearms from households in countries where there is political support for doing so would reduce suicide and violent crime.

                The fact that there is broad enough political support to do it in the first place is indicator of something pretty substantial though. What is changing in Australian society that makes this a priority in Australia? I’d argue that confounds the actual firearm role to some extent.

                I’d also argue that the very reason that Hawaii stands out in this scatter plot would also mean that buybacks of guns in Hawaii would (I expect) have basically no effect on suicides. Guns are plausibly a contributing factor, but many other fundamental factors are involved as well. If HI can have that many more guns than New England and still have similar death rates, it seems that those fundamental factors are pretty important.

              • Also, the suicide rate in australia does seem to show some kind of compensation, even if perhaps it doesn’t fully compensate.

                http://www.gunsandcrime.org/suichisty.gif

                That graph (which is ugly as heck) seems to show a long steady decrease in firearm rate, with no obvious changepoint in 96-97, but a definite jump in non-firearm suicides around 1997, and a generally increased level between 97 and 2003 or so.

                The more recent data suggests that male suicides are at an all time high in AU currently.

                http://www.theguardian.com/society/2016/mar/09/highest-australian-suicide-rate-in-13-years-driven-by-men-aged-40-to-44

                That’s my core hypothesis, that to the extent that firearms cause suicide deaths, firearms are *secondary* causes and the core issue is mental health, economics, family dynamics, societal effects etc.

              • Corey says:

                I’m assuming that the inconvenience imposed by these laws, no matter how trivial, will prevent some people from getting guns. As to what changed in Australia to make gun control a priority, I think this is the best explanation.

              • But, “some people” is not anywhere near the kind of reduction from the quoted article

                “Projected federal-level implementation of universal background checks for firearm purchase could reduce national firearm mortality from 10·35 to 4·46 deaths per 100 000 people, background checks for ammunition purchase could reduce it to 1·99 per 100 000, and firearm identification to 1·81 per 100 000”

                Even if we ignore homicide due to drug crime where these numbers a totally implausible on their face (none of those drug dealers abide by the existing laws) and look only at suicide where presumably most of the people are law abiding.

                And the suicide data in Australia may actually show the opposite of what people promoting their gun laws want it to. That spike in suicide in 1997,1998 and the fact that suicide has climbed to an all time high now in 2015 doesn’t suggest that the firearms laws have really been causally linked to reduced overall suicide.

                In any case, Australia always had high non-gun suicides, whereas here in the US many suicides are by firearm due to relatively common access. Common access that won’t change with those suggested laws (because firearm stock is unchanged).

                No, I think this is an interesting thing to study, and you have brought up some interesting background data, but I still stand by this article’s conclusions being full on baloney.

              • Corey says:

                I’m not defending the article; I don’t think one can draw worthwhile conclusions from an unconstrained regression with 25 predictors and 50 data points. I’m defending the proposition that “the availability of a convenient, rapid, painless method of killing one’s self is indeed a direct cause of suicide” and the implication that in a counterfactual world where guns are less convenient there are also fewer suicides.

              • Got it, I think I can get on board with at least the sign of the effect. ;-)

  3. David Winsemius says:

    I’ve seen repeated comments that this is a study involving only 50 data points. By that reasoning a controlled trial with only two study groups with an analysis based on event rates would have only 2 “data points”. Gun-mediated mortality rates involve much large numbers of persons at risk and much larger numbers of events. The “50” are actually strata in the analysis, _not_ data points. There are definitely issues to be raised about how many analyses can be legitimately conducted and what sort of correlations between the 29 categories of laws might exist and what sort of deaths (suicide versus family-member homicide versus non-related victim) are under consideration, but can we agree that this is not a study of “50 data points”?

    • Andrew says:

      David:

      No, I don’t agree with you. “50 data points” is shorthand for the fact that all the comparisons are being made at the state level. Yes, it’s possible to learn from a study with 50 data points, or from a study with 2 data points, or from a study with 1 data point. But as the number of data points decreases, you’re less and less able to estimate a model from data alone. Running an uncontrolled regression with 50 data points and 25 predictors is almost always a bad idea.

      • Curious says:

        Andrew:

        I agree that the r^2 will not be readily interpretable in such a scenario, but the means would be quite stable. No?

        • Curious says:

          And thus the parameter estimate similarly so.

        • Andrew says:

          Curious:

          No, the parameter estimates will be a disaster. It’s gonna be close to impossible to interpret the coefficients of these variables, each of which is controlling for all the others.

          • Curious says:

            I am working on a data modeling project in which the computer and servers on which I am attempting to model the data does not have enough memory to fully model the individual data points in a fully hierarchical fashion.

            To deal with this, I reduced by a date factor because the within day data are the most comparable relative to the between day data. This is not a time series, but different samples for each day. When I run individual regressions keeping the individual data points, the parameters are indeed somewhat lower, but the average is not lower by a huge amount than when run on the mean reduced data.

            I tried to deal with it in R by creating propensity score matches to reduce the size of the data, but missing data appears to be a problem for variables I consider critical to the matching process and I was not able to process the data in ‘mice’ as my computer does not have adequate memory to handle it.

            A suggestion with Stan perhaps? I have your BDA3 and McElreath’s book. Any suggestions would be greatly appreciated.

            • If it’s just your computer, do the model building and debugging on a reduced size dataset, and then when it is working, just spin up a preemptible google compute engine instance with enough RAM and spend a ~$5 to get the results.

              if it fundamentally is too big for any modern computer (ie. you’d need say 20,000 GB of RAM) then you probably need to rethink things.

              • Rahul says:

                My impression was that the $5 was not what deters people but the time and effort to get the code ported from your local machine to the cloud environment.

                Has Google Compute made this step trivial? Normally it used to be a heck of a job to set the environment right and get the right packages installed / compiled on the remote system. Especially if you used exotic packages etc.

              • Well, you can create your own image and upload it. Or you can install the software you need once you’ve got a standard image spun up. It doesn’t take more than say a couple of hours if you’re talking about R and a host of R packages like rstan and ggplot2 and plyr and RMySQL and soforth. And much of it is just telling things to install and then waiting in the background until they do. Network is obviously fast but if you’re uploading a big data set that might take a long time if your upstream internet connection is slow (like a cable modem or something).

                Once it’s set up, you can archive the system image so you can spin it back up as often as you want. That’s the part that’s been made really easy by Google et al.

  4. The Hawaii numbers are probably bogus (#4 below), but that’s not the biggest problem with the claims in Kalesan et al., or their post-publication “Response to Comments.”

    1. The biggest problem with this study is that the 25 state gun laws used as independent variables are highly correlated with each other (and with gun ownership levels). This multicollinearity, and the use of a multiple regression with relatively so many independent variables (25 variables for 50 data points), made spurious correlations all but certain. These problems are readily apparent when you consider that the study showed a 99.99% chance that requiring firearms trigger locks increases (yes, increases, with P 18 years to assess gun ownership and social gun culture performed in October 2013.”

    Estimates of gun ownership are all problematic for one reason or another, but all three major types of these estimates put Hawaii as one of the least gun-owning states:

    Surveys: to my knowledge, no large surveys are recent, and of course people lie to surveyors, but:
    BRFSS Survey Results 2001 for Nationwide Firearms, with 201,881 participants, has Hawaii at 50th out of 51. http://www.schs.state.nc.us/SCHS/brfss/2001/us/firearm3.html

    Background Checks: NICS background checks per 100,000 residents (Dec. 2008 – May 2010) puts Hawaii 49th out of 51 for gun purchases.

    Suicide: Fleegler et al. use suicide rations to put Hawaii as the state (of 50) with the lowest gun ownership rate. (“Percentage household firearm ownership was calculated by mean firearm suicides/total suicides (2007-2010) by state.”) – Fleegler, E., Lee, L., Monuteaux, M., Hemenway, D., & Mannix, R. (2013). Firearm legislation and firearm-related fatalities in the United States. JAMA Internal Medicine; 173(9):732-740. Retrieved from doi:10.1001/jamainternmed.2013.1286

    • The Hawaii numbers are probably bogus (#4 below), but that’s not the biggest problem with the claims in Kalesan et al., or their post-publication “Response to Comments.”

      1. The biggest problem with this study is that the 25 state gun laws used as independent variables are highly correlated with each other (and with gun ownership levels). This multicollinearity, and the use of a multiple regression with relatively so many independent variables (25 variables for 50 data points), made spurious correlations all but certain. These problems are readily apparent when you consider that the study showed a 99.99% chance that requiring firearms trigger locks increases (yes, increases, with P < .0001 and a most-likely increase by a factor of 3.90) firearms deaths. One might think that trigger lock requirements are helpful or that they are insignificant, but this is an absurd result, and it basically proves that there’s a problem with multicollinearity. There is no reason to think that the claimed decreases in deaths due to three gun control laws are any more real than is the almost 4-fold increase due to firearms trigger locks; indeed, these divergent results depend upon each other.

      2. The second biggest problem is that the study ignored race, even though racial composition all by itself shows a 69% r-squared (my calculation) against the states’ variation in homicide rates. While they claim to have also done so post-publication, their post-publication Figure 1 graph does not take race (or any other confounder except for gun ownership rates) into account.

      3. Ignoring Washington DC is also a factor. Gun control proponents tend to do so in such studies, because DC has a lower civilian gun ownership rate than any state, and a higher murder rate than any state. In their defense, DC is smaller and more urban than the 50 states, and it would be fairer to compare it to other cities than to other states.

    • (my first post was cut up somehow)

      4. Finally, as noted here, the key (if not essential) data point in Figure 1 in their post-publication “Response to Comments” is Hawaii –– and their estimate of gun ownership in Hawaii appears to be much too high. Kalesan et al. got their figures from a nationwide poll of 4000 people, which works out to an average of 80 people per state, probably less than that in Hawaii. Also, the poll was based on a nonrandom (largely self-selected) internet sample and had a 40% response rate.

      https://www.researchgate.net/publication/279629818_Gun_ownership_and_social_gun_culture
      “We used data from a nationally representative sample of 4000 US adults, from 50 states and District of Columbia, aged >18 years to assess gun ownership and social gun culture performed in October 2013.”

      Estimates of gun ownership are all problematic for one reason or another, but all three major types of these estimates put Hawaii as one of the least gun-owning states:

      Surveys: to my knowledge, no large surveys are recent, and of course people lie to surveyors, but:
      BRFSS Survey Results 2001 for Nationwide Firearms, with 201,881 participants, has Hawaii at 50th out of 51. http://www.schs.state.nc.us/SCHS/brfss/2001/us/firearm3.html

      Background Checks: NICS background checks per 100,000 residents (Dec. 2008 – May 2010) puts Hawaii 49th out of 51 for gun purchases.

      Suicide: Fleegler et al. use suicide rations to put Hawaii as the state (of 50) with the lowest gun ownership rate. (“Percentage household firearm ownership was calculated by mean firearm suicides/total suicides (2007-2010) by state.”) – Fleegler, E., Lee, L., Monuteaux, M., Hemenway, D., & Mannix, R. (2013). Firearm legislation and firearm-related fatalities in the United States. JAMA Internal Medicine; 173(9):732-740. Retrieved from doi:10.1001/jamainternmed.2013.1286

Leave a Reply