Research Design
The research approach used in the article by Lambert and Coomes (2001) is a quasi-experimental design.
The article evaluates the efficacy of the enterprise zone (EZ) program in Louisville, Kentucky, which was initiated in 1980 in response to “a rise in inflation, price hikes, poor monetary policy, and intense foreign competition” in this region (Lambert & Coomes, 2001, p. 171).
Its original goals were to stimulate economic activity, promote the general well-being of the inhabitants, and eradicate blighted urban districts. The authors used a quasi-experimental design to evaluate the EZ program by assessing its economic benefits before and after implementation, and comparing it with selected non-EZ regions.
The quasi-experimental design was appropriate as the study involved many extraneous variables, which made accurate measurement of the program’s benefits impracticable.
The authors cite three reasons why a pure experimental design would not have worked. First, the zone was expanded twice, in 1984 and 1986, to include other areas not originally included in the EZ (Lambert & Coomes, 2001).
Thus, in effect, the EZ had three distinct zones, which made it difficult to quantify the actual benefits of the EZ program using an experimental approach.
Moreover, a paucity of records could not allow one to attribute costs or benefits to a particular zone definition (Lambert & Coomes, 2001).
Second, a proliferation of economic projects in Louisville such as the airport project meant that less economic benefits could be directly attributed to the EZ program. Kentucky’s United Parcel Service (UPS) capitalized on the new infrastructure to expand their activities, which brought on board an additional 10,000 employees.
Additionally, other state economic development programs and legislations such as the Kentucky Industrial Development Act and the Jobs Act created favorable conditions for businesses to thrive. This attracted major investments from various companies, which created more jobs for the residents.
Third, the new investments in the EZ came with external costs (Lambert & Coomes, 2001). Firms from other regions moved their businesses to Louisville to take advantage of favorable tax regimes. However, this brought unintended external costs as firms that had incurred huge losses relocated to the EZ.
Although the effect of these extraneous variables was significant, they were beyond the control of the researchers. In this regard, a quasi-experimental approach was appropriate since this design does not require researchers to control for the effects of extraneous variables.
Moreover, since the EZ project was expanded twice (1984 and 1986) to include districts that were initially left out, a random assignment of the regions under the EZ program (experimental group) and areas outside the EZ region (control group) could not have been possible.
The project covered a period of 13 years, during which the EZ was expanded to include Louisville’s airport, an industrial park, Ford Motors factory, and some residential homes (Lambert & Coomes, 2001).
Thus, in this evaluation, a random assignment of the areas (urban districts) into the treatment and control groups was not possible, which ruled out a pure experimental design.
In this article, it is easy for one to draw causal inferences. The authors’ descriptive analysis seems plausible given the circumstances. They attribute the economic benefits in the EZ to the renovation of Louisville’s main airport and the UPS expansion, which created many jobs for the residents.
However, no benefits, in terms of improvement in quality of life, could be attributed to the EZ program. On the contrary, costs amounting to $55 million in tax exemptions were attributed to the EZ program.
Moreover, the EZ regions experienced more job losses compared to non-EZ regions like Newburg where 106 jobs were lost while the original EZ lost 6,890 jobs.
Based on these figures, it can be construed that the EZ program did not achieve significant benefits for Louisville residents, as the economic gains can be tied to other projects.
The quasi-experimental design used in this article involved an experimental group (the EZ definitions) and three controls (the Newburg neighborhood, the 1984 and 1986 additional portions, and the Jefferson County). The research units were not randomly selected into different groups; rather, they were treated separately.
Moreover, each unit had different independent variables, which not only affected the dependent variable (jobs created), but also served as rival hypotheses. Thus, the quasi-experimental design that the authors used was appropriate for this study given the circumstances.
Validity in Research
Validity describes the quality of the inferences or conclusions that a researcher draws from a study’s measure and designs.
In other words, a researcher’s conclusion or inferences can be said to be valid or not. In the article, the researchers, on analyzing the socioeconomic benefits of the Louisville’s EZ program, found that it had incurred losses of more than $55 million in tax expenditures.
They also found “a weak link between the private investments and the EZ incentives” (p. 178). These findings led to the conclusion that the program has not been effective in stimulating the intended economic activities in the region.
To determine if the study has conclusion validity, one has to assess the correlation between the variables (dependent and independent variables). From the author’s conclusion, one can infer that there is no relationship between the economic developments (dependent variable) and the EZ program (independent variable).
However, since there are several extraneous variables, including the economic benefits in the 1984 and 1986 portions, the author’s inferences cannot be validated. Since more investments came into Louisville EZ regions after the project commenced, it can be construed that the EZ program, in part, contributed to the economic development.
Moreover, the authors agree that the “post-1986 zone definition shows a smaller rate of out-migration, a smaller loss rate for occupied housing, better improvement in educational attainment, and greater growth in income” (p. 168).
This shows that the EZ program, in its three definitions, contributed to Louisville’s socioeconomic development. Thus, based on the information provided in the article, an assessment of the authors’ conclusion shows that it lacks validity.
Internal validity in the study relates to the causal construct. Just because the author’s investigation found a correlation between the infrastructure/development projects in the EZ region and economic growth (job creation), one cannot conclude that the investments caused the growth in jobs.
On the contrary, the EZ program might have caused both the investment increase and job growth. The article reports that the management of the EZ program noted that the program had created 11,800 jobs, saved 21,200 occupations, and attracted private investments worth $1.5 billion (Lambert & Coomes, 2001).
Though the authors dismiss these figures as incredible, a careful analysis reveals that the EZ program caused the economic outcomes.
This indicates that the conclusion that investment projects caused the outcomes does not take into account the internal validity of the study’s causal claim, as the program may have attracted investments into the zone, which then led to the study outcomes.
Another aspect of validity that is relevant to this study is the construct validity. In the study, the authors established a causal relationship between the private investments (the Ford Motor plant and the UPS) and job growth, but found no correlation between the EZ program and the outcomes.
In this case, construct validity relates to the implementation of the EZ program. The original EZ, which was commissioned in 1980, was expanded in 1984, to include Louisville airport, and in 1986 to cover targeted residential and industrial areas (Lambert & Coomes, 2001).
It can be argued that, because of the additional portions, the program was not implemented as intended. Thus, the authors should have assessed the three program definitions separately in order to have a better estimate of the program’s benefits. Another relevant aspect was the measures used in this study.
The authors measured job growth, cost of residential property, demographics, capital investments, and tax incentives. Thus, the evaluation ensured that they assessed what the study intended to measure.
External validity defines the correlation between the cause and the effect constructs. In other words, a study that has external validity allows generalization of its findings to other similar studies or contexts. The evaluation of the EZ program in this study found a weak correlation between the program incentives and job growth.
However, the authors did not give the implications of this finding for other EZ programs in other settings. The quasi-experimental approach adopted in this study makes it difficult to generalize the findings to other EZ evaluations.
Thus, the authors’ claims that the EZ program was not effective cannot be applied to other cities, as the extraneous variables are unique to Louisville.
Improvements that would strengthen the Study
The study’s measures and experimental design affected the quality of the evaluation. One of the challenges with the quasi-experimental approach relates to the choice of the control group. In the article, the authors used 4 controls, the Newburg, the 1984 portions, the 1986 regions, and the Jefferson County.
It was difficult to find a comparable control for the EZ zones. Moreover, the use of many controls affected the external validity of the evaluation.
To avoid this, the authors should consider comparing Louisville’s economic status before (pre-response measure) and after (post-response measure) the program (treatment) was implemented.
The interrupted time-series design would help the researchers to control for extraneous variables and develop a strong hypothesis.
Another way the evaluation can be improved is by using clear sampling methods. In particular, the inclusion criteria must be elaborate and clear. In the article, the authors chose Newburg, a non-EZ region, as the control, but did not indicate whether this zone had similar characteristics as the EZ definitions.
Thus, to enhance the study’s external validity, the authors should use formalized sampling methods when selecting the controls. Moreover, specified inclusion criteria can allow other researchers to identify and use the research units.
The third improvement that can be made on the study relates to program evaluation. Because the original EZ was expanded twice, each EZ definition should be evaluated separately. This will enhance the evaluation of the EZ program, as rival hypotheses can be identified and disapproved.
In its current state, it is difficult to attribute the outcomes to a particular causal factor. Thus, an assessment of the program in phases would help control for extraneous variables.
Reference
Lambert, E. & Coomes, P. (2001). An Evaluation of the Effectiveness of Louisville’s Enterprise Zone. Economic Development Quarterly, 15, 168-179.