Policy Analysis without Causal Identification: Gun Ownership and State Terror (Part 1)
Like many Americans, we are saddened and sickened by the recent shootings in Aurora and Oak Creek. Like many Americans, we have also been discussing (among ourselves, and with friends) the prevalence of firearms in the United States. There are more guns in America per capita than in another other country in the world. And we also live in a very violent society, especially relative to our level of socioeconomic development. It is natural to wonder if guns are partially responsible for this violence, and to think about what the associated costs of gun control might be.
This is a two-part post in which I will tackle one common defense of America’s liberal firearms laws: popular defense against state terror. A broader methodological point will be to show that even without causal identification, it is easy to produce findings that should shape our understanding of public policy, which is contrary to the idea that only causal identification produces policy relevance. (More on that in the next post.)
The Argument
Our good friend Matt Glassman is one very articulate defender of (some forms of) gun ownership rights. One argument that Matt finds particular compelling is that an armed society can protect itself against tyranny or state terror. Matt puts it best:
I swear, you mention the idea that gun ownership protects people from state tyranny, and pretty much every liberal immediately makes some joke about Joe Sixpack stopping a U.S. military tank with his handgun. But I think this precisely misreads the situation on two levels. The first is the factual: anyone who has ever watched the youtube videos from Iran knows that the tyrannical state doesn’t maintain its grip by employing tanks everywhere; there’s simply not enough money to do that. Instead, it relies on thugs and other ill-trained militia to terrorize citizens with knives, clubs, and small arms. You watch the video of a man being assaulted on his own front lawns by one or two state thugs with knives and clubs, and it become self-evident that such practices simply would not be possible in a society with as many homeowner firearms as the United States.
It just so happens that we here at Indolaysia have lived under authoritarian rule—the soft authoritarianism of contemporary Malaysia, where the state isn’t tyrannical but where state terror can, and has, cropped up from time to time. Let’s not pretend that we faced anything like a threat to our physical safety from the Malaysian state. But some people in Malaysia have. Our sense is that arming Malaysian civilians would not protect them against the terror that they face. But we could be wrong.
The Data
One way to check is to collect a broad range of data on state terror and gun ownership. Is it true that countries with lots of guns tend to have less state terror? First let’s look at the data on private gun ownership. I gathered this from the Small Arms Survey, using 2007 data.
Private Gun Ownership per 100 people, 2007 data
The U.S. really does have a lot more privately-held firearms than any other country, relative to our population.
But is this an essential part of our defense against state tyranny? If so, we’d expect that overall, countries with lower levels of gun ownership are more subject to the tyranny of the state than than countries with high levels of gun ownership. Let’s compare the (natural log of) privately held arms per 100 people with some standard indices of state terror. Take first some rankings developed from Amnesty International. These are 2000-07 averages of a yearly index that ranges from 1 – 5, with 1 the “worst” for state terror and 5 the “best.” (Data are from the Political Terror Scale.) To illustrate another important distinction, I’ve graphed the democracies in blue, and the dictatorships in red, using the Cheibub and Gandhi binary regime classification.
Gun Ownership and Political Terror
The overall trend line (in black) shows that there’s a positive relationship overall between gun ownership and freedom from state terror. But these data also don’t make the U.S. look very good in the comparative context. We aren’t as free from terror as we think we are. So let’s use data collected by the State Department itself about political terror worldwide.
Gun Ownership and Political Terror (2)
Again, a modest positive relationship between gun ownership and freedom from state terror. We can repeat this further, if we like, for an index of civil liberties.
Gun Ownership and Civil Liberties
The same pattern continues to hold. Now looking at these three figures, we can see a couple of conclusions.
- There is a modest positive relationship between gun ownership and freedom from state terror. This is important.
- There is also clear evidence that democracies terrorize less than dictatorships. (The blue countries tend to be further to the right than the red ones, on average.)
- The United State is a major outlier. There are plenty of examples of countries that are equally as free from state terror but which do not have nearly the same level of private gun ownership.
Taken together, this makes me skeptical that broad ownership of guns by citizens is much of a defense against state terror. It is hard to conclude that the level of gun ownership that the United States sees today is necessary to defend against tyranny. We can even repeat the exercise for other more precise indicators of state terror.
Gun Ownership and Six Indicators of State Terror
In fact, you look at these indicators and you see that the U.S. doesn’t even really do that well, even with all of our guns.
A Closer Look
But does gun ownership help, on the margins? Let’s run some simple regressions.
|
|
||||||
| Pol Terror (Amnesty) |
Pol Terror (State Dep.) |
Civ. Lib. (F. House) |
||||
|
|
||||||
| Ln(guns) | 0.166** | -0.0525 | 0.250*** | -0.0514 | 0.431*** | 0.0439 |
| (3.22) | (-0.95) | (4.50) | (-0.93) | (4.45) | (0.60) | |
| Democracy | 0.417** | 0.399** | 2.366*** | |||
| (3.27) | (3.22) | (14.41) | ||||
| ln(GDP per capita) | 0.335*** | 0.455*** | 0.357*** | |||
| (5.80) | (8.11) | (4.80) | ||||
| constant | 3.118*** | 0.419 | 3.061*** | -0.526 | 3.790*** | 0.116 |
| (29.83) | (0.97) | (27.09) | (-1.24) | (19.08) | (0.21) | |
|
|
||||||
| N | 158 | 152 | 167 | 161 | 168 | 162 |
|
|
||||||
| t-statistics in parentheses. * p < 0.05, ** p < 0.01, *** p < 0.001. | ||||||
Consistent with the graphical results, there is an unconditional correlation between gun ownership and freedom from political terror. This correlation is statistically significant. But it disappears once we control for two other standard determinants of rights: democracy (measured as before) and economic development (measured as the log of GDP per capita, 2007-09, from the World Development Indicators).
The conclusion that this suggests is that the United States is relatively free from state terror because it is a developed democracy, not because we allow our citizens broad freedoms to arm themselves.
Of course, these are not estimates of the causal effect of gun ownership (or anything else) on state terror. These are conditional correlations, and there are plenty of reasons why we might believe that the causal relations here are more complicated than what this discussion has implied. More in the next post on what those threats to causal inference are, and how we ought to go about drawn policy recommendations from messy conditional correlations such as these.
Replicate!
Finally, to recreate these figures (and more) and to recreate these analyses (and more), you can download my Stata .do file here. Just put it in a folder with the path c:\data\guns_and_terror, make sure you’re connected to the internet, and run it.
Holy cow, this is some sweet social science! Well done, Tom.
One theoretical thing worth pondering is the relationship between democracies and gun laws. I’m not some NRA-freak gun guy, but I can guarantee you that any such person’s first critique of your model would be something like this: democracy and loose gun control are not independent of each other and, in fact, the latter helps sustain the former.
I will not go so far as to endorse that as a reason for what we see in the graphs. But I also won’t reject it as possible; if loose gun laws do have an affect on state terror, almost by definition they should have a marginal effect on keeping a democracy from slipping into a tyranny.
In any case, I think it’s worth pondering — big picture — why authoritarian regimes generally cannot tolerate loose gun laws. Because I think it’s hard to answer that question without coming around to the idea that loose gun laws do have a meta effect against the state.
Looking forward to part II.
m
That’s a super comment, Matt. Thanks for reading and thinking.
This is going to be a big part of the next post: it’s really hard to disentangle the causal effect of gun ownership on state terror
GO -> ST
if you think that maybe
DEM -> GO -> ST
and also that
DEM -> ST
There’s no easy way around it. But I think that these results are about changing our priors about how the world really is. With these results, I am now less confident that gun ownership is actually a protection against tyranny than I used to be. It’s possible, but we have to invent some sort of reason why the data look like this to make sense of these results while maintaining the view that gun ownership is a defense against tyranny.
Great post, Tom. It’s nice to see you use all of the data. Often these discussions devolve into cherry picking country examples (e.g., Canada or Switzerland).
Maybe I should wait to read Part II, but I think Matt’s point and your response are spot on. Getting at the causal chain is going to be difficult. Another possibility is that:
GO -> DEM -> ST
Bringing in time would help get a sense of how these things have unfolded in countries. For instance, do we see ST go down and DEM go up *after* GO goes up? This of course still makes assumptions about no unobserved heterogeneity, but seems like you would need to observe this pattern if GO is a necessary condition for DEM maintenance.
Thanks for reading and commenting, Kevin. You’re totally right—the causal ordering issue is a big deal too. Then consider that
ST -> GO
and the problems really start to mount. Temporal variation is one way to start looking, but I’m skeptical that even decade-averaged data is going to pick up the variation that we want (esp. given the unobserved country-level heterogeneity that will contaminate any sort of temporal analysis).
I’ll have to think hard about my second post now….
Reading all this, I’m both glad I have friends who can crunch numbers like this, and even more glad that I am a cultural historian who doesn’t have to! Nice stuff, Tom!
Thanks for reading, Charley. The secret is that I actually ENJOY this stuff.
[...] analysis without causal identification: gun ownership and state terror” in two parts: part 1 and part 2. Includes .do file to replicate his [...]
Tom, thanks for the analysis. The tricky thing I find about an in depth look like this are the types of weapons owned and used in the respective societies. In the US, a majority of the gun crimes are perpetrated by handguns which would be of little use in defending a populace from a government enemy, whether foreign or domestic. My point is, rather my question – is there any data out there that could lend itself to a similar comparison based on firearm types?
That is certainly a good question. I’m not sure what sort of data is out there that would allow you to disaggregate across firearms types.
I would say, though, that your observation is bad news for the argument that gun ownership protects Americans from state terror. We might support handgun ownership for lots of reasons, but it’s probably not because it keeps us safe from tyranny.