Research question: Did Tennessee’s 2014 legal change to allow unlicensed individuals to carry firearms in their vehicle lead to an increase in gun thefts from vehicle?
1.Unit of analysis
Since gun laws are typically passed at the state-level, the units of analysis in this study are states.
2. Identify the main variables, operational definition, level of measurements, and address threats to reliability and validity of the measures:
Your dependent variable,
At least one independent variable, possibly more if appropriate.
Dependent variable:
Gun thefts from vehicles
Operational definition: The number of incidents reported to the police that involve theft of property inside a vehicle where at least one firearm was stolen.
Level of measurement: The dependent variable will be a ratio variable because the distances between the attributes of the variable is meaningful and a value of zero on this variable is meaningful (it indicates the absence of gun thefts from vehicles).
Threats to measurement validity:
Face validity: This measure has face validity because it provides a logical representation to measure gun thefts from vehicles.
Criterion-related validity: This measure is an official measure of gun thefts from vehicles and therefore is generally accepted as a valid measure of the construct from theft from vehicles. Therefore, our dependent variable has criterion-related validity.
Multiple measures: One way to examine whether the variable captures changes in gun thefts from vehicles may be to compare it the number of insurance claims filed for stolen firearms. If the two measures are correlated, we can expect that our measure has construct validity.
Threats to measurement reliability:
In order to address reliability, we have to make sure that the recording practices of the thefts from vehicles does not change over time. Official crime statistics may change over time because of these recording practices or other legal changes.
Independent variables:
Implementation of a law to carry a gun inside vehicles without a permit
Operational definition: A state passes and implements a law that allows citizens to carry a weapon in their vehicles even when they are not allowed to carry guns in public.
Level of measurement: This independent variable is a nominal variable because it labels each state as having this type of law or not.
Threats to measurement validity:
Criterion-related validity: There may be a concern that even though the law is passed, it takes time for law enforcement to adjust to the change, which would mean that the introduction of the law is not best measure of when people are actually allowed to carry guns in their vehicles. To make sure that our measure is valid, we will compare the timing of the law with the change in the number of arrests for illegal gun carrying in vehicles. If our measure is valid, we should see that arrests for gun possessions in vehicles decrease substantially after the law is passed.
Threat to measurement reliability:
It is possible that our measurement of the independent variable lacks reliability, especially in states that did not pass the specific law that was passed in Tennessee. Other states might have passed laws that are similar but not exactly worded the same. To ensure that our measure is reliable, we will have several researchers to review the legal codes of the states included in our study and test the interrater reliability between the researchers. If they agree on which states have the specific laws and which states does not, we will ensure that the measurement is reliable.
Urban settings (only for research design #2)
3.Describe two potential research designs to answer your question. For each research design, you will need to…
Research design #1
Describe the design, the groups, and the assignment to the groups: The first potential research design for this study is an interrupted time-series design with a non-equivalent comparison group (see Figure 1). The treatment group will be the state of Tennessee, which is the only State implementing the law in 2014. The comparison group will be other States that did not change their laws at any point in the period before or after the Tennessee law was passed in 2014. The comparison group will be made of 13 other states that did not change their laws during the study period and had a comparable way of measuring the dependent variable. Many states did not report sufficient information about gun thefts from vehicles to the FBI, and therefore they were not included in the study. Comparison states were comparable to the treatment state in the restrictiveness of their gun laws. In this study, it is impossible to randomly assign States to the treatment which makes this design a quasi-experimental design.
Describe how and when data will be collected: For the treatment group, we will measure the dependent variable every month for several years before and several after the introduction of the independent variable (the implementation of the law). The measurement process will occur at the same time and over the same time period for the comparison group. For both groups, we will measure the dependent variable each month from January 2009 to January 2018.
Describe how the independent variable is introduced: The independent variable is introduced only in Tennessee by the implementation of the gun carrying law in July of 2014. The independent variable will be measured by coding the legal codes of all the states included in the study and assess whether the state implemented the gun carrying law and the exact month the law was implemented.
Threats to internal validity (three addressed by the design):
History: The threat to internal validity called history might lead researchers to wrongly conclude that their independent variable is causing a change in their dependent variable. In fact, the change is due to an event of social or historical significance occurring at the same time outside the experimental setting. For example, in this study, gun thefts may increase in Tennessee because of a national tragedy such as the Sandy Hook mass shooting that occurred during the study, leading people to be desperate to acquire guns for protection. In this study, we are guarding against this threat to internal validity by including a comparison group. We will still be able to test the effect of the law on gun theft in the treatment group even if such an event occurs because the event should influence both the comparison and treatment group equally. The remaining difference in the groups after the intervention could still be attributed to the change in the law.
Instrumentation: The threat to internal validity called instrumentation might lead researchers to come up with the wrong conclusion regarding the causal relationship between their two variables because the change in the dependent variable is due to a change in the way it is measured before and after the independent variable is introduced. In this study, this treat to internal validity may occur if either the law enforcement or the FBI changes the way gun thefts from vehicles are reported. While it is impossible to know for sure that all agencies recorded thefts from vehicles consistently over the study period, we know that the FBI did not modify their rules and regulations related to these types of crime during the study period.
Ambiguous causal time order: The ambiguous causal time order threat to internal validity occurs when the causal order is reversed meaning that a change in the dependent variable is actually causing the change in the independent variable. While it is technically possible that the increase in gun thefts from vehicles actually caused Tennessee law makers to pass a law allowing civilians to carry guns in their cars, there is no evidence that this was a reason motivating the passage of the law. Even if that was the case, the multiple pretest measures of the dependent variable guard against this potential threat to internal validity.
One threat to internal validity that might be still an issue:
Testing: The threat to internal validity called testing might lead researchers to be wrong about their conclusion of a causal relationship because the change in the independent variable itself may lead to a change in the way the dependent variable is measured before and after the treatment. In this study, this threat to internal validity may occur because making it legal to carry guns in a vehicle might make people more likely to report a stolen gun after the law compared to before the law. Before the law, reporting a stolen gun from your vehicle might have been perceived by victims as essentially admitting that they were illegally carrying a weapon in their cars. Therefore, it is possible that the law causes and increase in thefts from vehicle simply because citizens feel more comfortable to declare that a gun was stolen now that the practice is legalized. In this case, it is difficult to completely rule out this alternative explanation.
External validity: Threats to external validity are possible in this study. For instance, because the law was passed in only one state, it is unclear whether a similar law would have the same effect in other states. Tennessee is a state that is not very restrictive in terms of gun laws, which means that guns may be more easily available than in a more restrictive state. It is possible that allowing guns in cars in a state with more restrictive gun laws, or with more urban centers, might see an even greater increase in gun thefts from vehicles.
Figure: see Figure 1.
Repeat all steps necessary to describe Research design #2 (at least steps 1 through 7 under research design #1, but maybe you need to modify your variables or add an independent variable)
Best design and ethics: The interrupted time-series design with nonequivalent comparison group will be the better design for this study. While there are threats to external validity associated with the design, the interrupted time-series design with nonequivalent comparison group does provide protection against several threats to internal validity such as ambiguous causal time order, history, and instrumentation.
In this study, concerns associated with revealing the identity of participants are minimal, since the unit of analysis is at the state level. Nevertheless, while crime incident statistics are anonymous, looking at such a specific type of crime may accidentally reveal the identity of victims for those who are aware of the circumstances of the crime. To ensure that this is not possible, we will not report data on any time period where less than 10 incidents are reported to the police.
No consent or deception is necessary in this study because we are not engaging in human subject research.
The only potential violation of ethics in this study would be related to analysis and reporting. Since we will be using publicly available data in this study, it will be possible for independent researchers to replicate our study. We will make our calculations and analyses publicly available to ensure that our research is fully transparent and that the researchers involved did not manipulate the data or make up the results.
Appendix
Figure 1. Interrupted time-series design with nonequivalent comparison groups
OO…OOXO O…OO
——————————————————————————————————-
OO…OOOO…OO
t1t2…t66t67 t68t69t114t115
O = Monthly number of gun thefts from vehicles
X = Implementation of the Tennessee law
tn =Time before and after the law
