Chapter 9 throws light upon an important aspect of non-randomized designs, which is bias. As bias significantly diminishes the validity of results and questions the impact of programs, the knowledge and control for it are crucial. Some types of bias may be easier to eliminate than others (Rossi, Lipsey, & Freeman, 2003). As such, ex-post measurements that compare the changes in the naturally increasing or decreasing variable are often subject to statistical errors and could be rather uncomplicated to find and avoid. Some factors like attrition, for instance, could be somewhat more difficult to control for, and statistical tools would have to be used to even out the results (Patten, Newhart, & Newhart, 2017). Attrition is a withdrawal of a participant from the study. Overall, all forms of bias in one way or the other are significant barriers towards the fair and effective assessment of program outcomes, which produces the need to review the design carefully and use only valid and frequently used measures.
In terms of quasi-experimental designs that could be instrumental in situations when complete randomization is not possible, there are also a few important insights. Firstly, the main takeaway is that such studies could be subject to subtle yet significant differences in intervention and control groups which require researchers to take extra precautions against selection bias (Rossi et al., 2003). In light of this, matching participants and variables and equating them statistically becomes critical for producing adequate assessment results. Among the most viable and error-prone designs within the quasi-experimental domain are regression-discontinuity types (Orcher, 2016). The regression-discontinuity analysis is a cutoff pre-post study design that allows measuring case and effect connections. Despite the fact that it is not always applicable, it is capable of producing meaningful results if researchers could collaborate with program executives to engage in a systematic division of participants. This allows for a decreased chance of not controlling for important independent variables, as program personnel often have more knowledge of them. Yet, it is also important to mention that there might be a limited number of programs eligible for cutting-point designs, as participation criteria are rarely strictly defined.
Another valuable takeaway from the chapter is that randomized controlled types of study are still the preferred choice for an accurate assessment as quasi-experimental ones are rather difficult to devise in a bias-free manner (Rossi et al., 2003). Nonetheless, in cases when it is impossible, quasi-experiments could be rather useful if certain factors are taken into account. Thus, researchers need to control for selection bias as the participants chosen for a study are one of the influential core factors that affect the quality of results. In addition to that, there is a need for developing proper methods of group equation as it directly leverages the fact of registering the significant change.
References
Orcher, L. T. (2016). Conducting research: Social and behavioral science methods. New York, NY: Routledge.
Patten, M. L., Newhart, M., & Newhart, M. (2017). Understanding research methods: An overview of the essentials. New York, NY: Routledge.
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2003). Evaluation: A systematic approach (7th ed.). Thousand Oaks, CA: SAGE Publications, Inc.