Original Article By Maggie Koerth for FiveThirtyEight (part of ABC‘s online presence)
In the first month of 2023, 25 people lost their lives in four mass shootings in California over just eight days. It’s a grim statistic, made all the more distressing when you consider the fact that California has one of the lowest gun death rates in the entire country. This is what a safe state looks like.
California also has some of the strictest gun control laws in the country. And in the aftermath of those four mass shootings, new House Speaker Kevin McCarthy — who represents a district in southern California — took the opportunity to poke at the state’s firearms restrictions, saying in a press conference that federal gun control legislation would not be an automatic response to these tragedies because such laws “apparently … did not work in this situation.”
So, did California’s gun laws succeed at making it one of the safest states … or did they fail to stop a string of mass shootings? Questions about the efficacy of gun laws have gotten easier to answer in recent years as changes to federal policy have helped to bring money and people back to the field of gun violence research. But decades of neglect mean there are still lots of blank spaces — policies that don’t yet have good quality data backing them up. A recent report from the Rand Corporation that reviewed the evidence behind a variety of gun policies found just three that were supported by evidence that met the report’s quality standards.1
That fact, however, doesn’t mean other gun laws don’t work — just that the research proving it doesn’t yet exist. Scientists I spoke to saw it as an “absence of evidence” problem, stemming from long-standing, intentional roadblocks in the path of gun violence research. Even the authors of the Rand report say lawmakers should still be putting policies aimed at preventing gun violence into practice now — regardless of what the science does or doesn’t say.
“I think that the goal of the lawmaker is to pick laws that they have a reasonable hope will be better than the status quo,” said Andrew Morral, a senior behavioral scientist at the Rand Corporation. “And there’s lots of ways of persuading oneself that that may be true, that don’t have to do with appealing to strict scientific evidence.”
California doesn’t just have some of the nation’s strictest gun laws and lowest gun death rates, it’s also maybe the best state to study gun laws in, said Dr. Garen Wintemute, director of the Violence Prevention Research Program at University of California, Davis Medical Center. That’s because of both the way the state makes data available to researchers and its willingness to work with researchers to further the science. Wintemute is currently part of a team that is working on a randomized controlled trial of one particular California gun law — an initiative that tracks legal gun owners over time and dispatches authorities to remove their weapons if those people later break a law or develop a condition that would make them ineligible to own guns in the state.
It’s hard to oversell what a big deal this is. Frequently referred to as the “gold standard” of evidence-based medicine, randomized controlled trials split participants randomly (natch) into groups of people who get the treatment and groups that don’t. Because of that, it’s easier for researchers to figure out if a medication is actually working — or if it just appears to be working because of some other factor the people in the study happen to share. These kinds of studies are crucial, but almost impossible to do with public policy because, after all, how often can you randomly apply a law?
But California has been willing to try. It took cooperation from many different levels of state leadership, Wintemute said. The government was always going to slowly expand this particular program statewide, but in this case legislators were willing to work with scientists and randomize that expansion across more than 1,000 communities, so that some randomly became part of the program earlier and some later. When the study finally concludes, researchers will be able to compare these two groups and see how joining the program affected gun violence in those places with a high level of confidence.
Most of the time, however, the scientists who study gun laws aren’t working with the kind of research methodology like this that produces strong results. Morral, along with his Rand colleague, economist Rosanna Smart, have reviewed the vast majority of the research on gun control policies done between 1995 and 2020. Their research synthesis found that a lot of what is out there are cross-sectional studies — observational research that basically just compares gun violence statistics at one point in time in a state that has a specific law to those in a state that doesn’t. That type of study is prone to mixing up correlation and causation, Smart said. There could be lots of reasons why California has lower rates of gun violence than Alabama, but studies like this don’t try to tease apart what’s going on. They end up being interpreted by the public as proof a law works when all they’ve really done is identified differences between states.
The Rand analysis threw out these kinds of studies and only looks at research that is, at least, quasi-experimental — studies that tracked changes in outcomes over time between comparison groups. Even then, the analysis ranked some studies as lower quality than others, based on factors such as how broadly the results could be applied. For instance, a study that only looked at the effects of minimum age requirements for gun ownership in one state would be ranked lower than a study that looked at those effects in every state where a law like that existed.
Stand-your-ground laws “appear” to increase firearm homicides.
Following these rules, the Rand team found just three policies that have strong evidence supporting outcomes — and two of these are about the negative outcomes of policies that increase gun access. Stand-your-ground laws, which allow gun owners to use deadly force without trying to leave or deescalate a situation, appear to increase firearm homicides. Meanwhile, conceal-carry laws, which allow gun owners to carry a gun in public places, appear to increase the number of all homicides and increase the number of firearm homicides, specifically. The only laws restricting gun ownership that have this level of evidence behind them are child-access prevention laws, which have been shown to reduce firearm suicide, unintentional self-injuries and death, and homicides among young people.
That makes gun control laws seem flimsy, but it shouldn’t, Morral said. Instead, the lack of evidence ought to be understood as a product of political decisions that have taken the already challenging job of social science and made it even harder. The Dickey Amendment, first attached to the 1996 omnibus spending bill, for example, famously prevented the Centers for Disease Control and Prevention from funding gun violence studies for decades. A new interpretation of that amendment in 2018 changed that, but Dickey wasn’t the only thing making it hard to study gun violence.
Instead, the researchers told me, the biggest impediment to demonstrating whether gun control policies work is the way politicians have intentionally blocked access to the data that would be necessary to do that research.
“So for instance, the federal government has this massive, great survey of behavioral risk indicators that they do every year in every state,” Morral said. “And you can get fantastic information on Americans’ fruit juice consumption as a risk factor for diabetes. But you can’t get whether or not they own guns.” Not knowing gun ownership rates at the state level makes it hard to evaluate causality of some gun control policies, he explained. “And it’s not because anyone thinks [gun ownership] is not a risk factor for various outcomes. It’s because it’s guns.”
The missing data problem also includes the 2003 Tiahrt Amendment that prevents the sharing of data tracing the origins of guns used in crimes with researchers, said Cassandra Crifasi, co-director of the Center for Gun Violence Solutions at Johns Hopkins University. “So now all we can see are these sort of aggregate-level state statistics,” she said. “We can no longer look at things like, when a gun is recovered in a crime, was the purchaser the same person who was in possession of the gun at the time of the crime?”
Recently, researchers have even been missing basic crime data that used to be reported by the FBI’s Uniform Crime Reporting program. Law enforcement agencies and states were supposed to be shifting to the relatively new, much more detailed National Incident-Based Reporting System, but the transition has been a catastrophe, with some of the biggest law enforcement agencies in the country not yet making the switch because of financial and logistical complications, Smart said. “The FBI has not been able to report for the last eight quarters whether homicide rates are up or down,” Morral added.
But much of the data that’s not available at a national level is available in California, Wintemute said. “Unlike researchers in any other state, we have access to individual firearm purchaser records,” he told me — the very data the Tiahrt Amendment blocks at the national level. “We do studies involving 100,000 gun purchasers, individually known to us, and we follow them forward in time to look for evidence of criminal activity or death or whatever the outcome might be that we’re studying,” Wintemute said.
Unfortunately, because the data is only available in California, the results of those studies would only be applicable to California — making it data that wouldn’t be considered high-quality in the Rand report. Wintemute can demonstrate if a policy is working in his home state, but not whether it works in a big, broad, existential sense. It wouldn’t count towards expanding the number of policies Rand has found evidence to support. This is something researchers like Crifasi see as a flaw in the Rand analysis, but it’s also a reason why Morral and Smart don’t think the evidence-based policy is a good standard to apply to gun control to begin with.
It’s useful to know what there is evidence to support, Morral said. “But we don’t at all believe that legislation should rest on strong scientific evidence,” he said. Instead, the researchers from Rand described scientific evidence as a luxury that legislators don’t yet have.
“There’s always gonna be somebody who’s the first person to implement the law,” said Smart. “And they’re going to have to derive their decision based on theory and other considerations that are not empirical scientific evidence.”
Maggie Koerth is a senior science writer for FiveThirtyEight. Part of ABC.
