Authors: Jennifer L. Doleac and Anita Mukherjee

Reviewed by: Carrie Wade and Chelsea Boyd

Harm reduction approaches to manage the risks or consequences from opioid use have already helped reduce rates of infectious disease transmission from injection drug use and overdose deaths. They have also helped reduce the stigma associated with drug use, which is a major barrier to treatment and care. These are excellent developments. However, whenever the consequences of risky behavior are diminished (even when they are purposefully reduced to achieve a net-benefit), there is a chance that the risky behavior may escalate. This is known by economists as a “moral hazard.” For example, during bad weather, drivers with All Wheel Drive (AWD) often take more risks than drivers without the feature. This is likely because drivers with AWD believe they are protected from the increased risk of driving in snow, so they may not adjust their behavior to account for the adverse conditions.

With respect to harm reduction, a recent study by Jennifer L. Doleac and Anita Mukherjee argues that expanded naloxone access increases opioid abuse and crime, while having no effect on overdose mortality. Such increases, they argue, occur because naloxone prolongs the lives of people who use opioids (allowing them to continue drug use) and increases the likelihood that people may use in a more risky manner, simply because naloxone is available if they overdose. Furthermore, because naloxone may increase the number of people who use opioids, theft and other crime may increase to support use.

While such arguments may seem plausible, albeit distasteful, they are simply contradictory to the existing body of research on the expansion of naloxone access. And, while it is true that sometimes evidence emerges that successfully disproves years of previous research, this is not one of those cases, as the study’s conclusions are based on questionable methods, a narrow dataset and far too many questionable  assumptions.  

With respect to the study’s methodological problems, a primary one is that each of the associations presented in the paper are derived from many, diverse data sources that were not designed to be combined. For example, data is derived from (among others) Google search trends, The National Incident-Based Reporting System, The Healthcare Cost and Utilization Project, a restricted-use dataset on opioid overdose mortality from the Centers for Disease Control and manually collected data on state-level naloxone access laws.

Combining datasets is problematic because each of their data collection methods may differ significantly. In this case, it is of particular concern because the authors of the study failed to describe how the large number of datasets compared methodologically, and (worse yet) they provide no description of the methods used when collecting the data they personally contributed to the study. Additionally, many of the associations they derived were from proxy variables intended to represent a different exposure than the variable measured. Essentially these types of substitutions create a scenario similar to replacing a key ingredient in a recipe; the result may be good or bad, but it’s definitely not the same as the original. In this case, their choices of proxies are often not ideal and the choice to restrict their analysis to urban areas with populations greater than 40,000 make it difficult to generalize the findings to other areas of the United States.

The study’s findings are also plagued by its empirical strategy. For example, to determine how expanded naloxone access impacts behavior, the authors compared jurisdictions that adopted naloxone access laws against those that did not. In so doing, they used a differences-in-differences (DD) model, commonly used in program evaluation to evaluate outcomes resulting from an intervention. Such a model requires the use of longitudinal data (data collected from the same subjects at multiple time points over a set period), and relies on several assumptions, most importantly, that in the absence of treatment, the difference between the treatment (areas with expanded naloxone access) and control (areas that do not have expanded naloxone access) groups remain constant over time. This is known as the “parallel trends assumption.”

However, since opioid overdose mortality rates were not increasing uniformly across jurisdictions during the study period, it’s unlikely this assumption was valid and likely that it led to inaccurate results. This is because the parallel trends assumption is the most important assumption made in the DD model. Therefore, if the assumption is not valid for the data, the resulting estimate will be biased and in some cases, extremely so. Since it is likely that naloxone expansion is happening in response to rapidly increasing overdose deaths or the increasing prevalence of opioid use, the intervention group is very likely changing at a more rapid pace than the control group. Unfortunately, to date, research to confirm or refute this hypothesis is lacking and although the authors attempt to address this concern, it is unlikely that their model adjustments sufficiently control for the differences between treatment and control groups.

Another concern is that the authors fail to correct for autocorrelation across time periods. Put simply, this is when the value of an observation in a time series is influenced by the value of a previous observation. Such a failure to correct can result in erroneous conclusions up to 85 percent of the time, as it biases the estimate at each subsequent time point and results in the erroneous conclusion that there is an association between variables. Doleac and Mukherjee suggest that the results of their analysis are robust, and provide several analyses in support. However, without evaluating the impact of autocorrelation, there is no way to determine the validity of the results.

Finally, the follow-up period after the implementation of naloxone access law was no more than 12 months for any measure and, in some cases, was only six. It is questionable if such short follow-up can accurately reflect the impact of the law. After all, there is no such thing as perfect implementation or information in public health, and often those most in need of intervention access do not learn about available programs until long after original implementation.

All of this is to say that while, on the surface, Doleac and Mukherjee present a fairly convincing argument that naloxone access laws may not have the desired effects, their results depend on so many assumptions that it is difficult to trust their findings.

Perhaps more disturbing, however, than the questionable findings themselves are the potentially dangerous takeaways that may be extrapolated from them. We should never condemn a policy that provides access to a lifesaving treatment based only on the possibility of negative externalities like property crime. Even if, under the many assumptions of this model, the overdose mortality rate remained stable, no one can argue against the fact that naloxone saves individual lives. And accordingly, increased availability may be very literally be the difference between life and death.