Introduction
Evaluation is a complex, resource-consuming, and nuanced process, yet it remains exceptionally important to determine whether certain programs are working or develop strategies to increase their effectiveness. An organization can only make educated decisions and create accurate projections for the future if it manages to evaluate existing programs and initiatives successfully. Thus, over the past decades, one can notice a growing trend of utilizing evaluation to solve immediate problems, improve operations, as well as implement efficient strategies in the long run. The primary challenge evaluators face while trying to assess the significance of a certain program is that each project has unique requirements. Therefore, it is their task to determine which standards to apply for the specific evaluation at hand. The purpose of this essay is to examine the evaluation process to identify factors influencing the choice of standards and application strategies used by evaluators.
Defining Evaluation
To delve deeper into the evaluation process, it is crucial to define what “evaluation” actually is. Milstein and Wetterhall (2021) regard it as “the systematic investigation of the merit, worth, or significance of an object or effort” (para. 6). According to Piccioto (2011), evaluation is not merely a process of collecting empirical data but a multi-level effort to gather contextual information as well. It is important to note that evaluation aims to address key concerns related to the project, including its efficiency, process, structure, impact, and so on (Schultes et al., 2018). Based on these findings, various stakeholders then make critical decisions about the initiative’s planning and implementation.
Evaluators focus on a range of aspects that constitute a specific project. These include the needs of the target audience, the resources invested, the design methodologies, the effectiveness of the program, as well as the primary and secondary benefits or drawbacks (Day, 2006). All of these diverse elements constitute an overall set of valuable practices to facilitate intended change. Although these definitions manage to provide clarity as to what evaluation involves, it is crucial to note that, historically, the concept of evaluation has changed over time (Hogan, 2007). The field continues to welcome new approaches and methods to address increasingly diverse and complex projects.
Evaluation Process
Specific aspects of program evaluation can determine which standards evaluators will end up utilizing. According to Milstein and Wetterhall (2021), the evaluation process begins by answering the most critical questions. Thus, evaluators identify the object of evaluation, criteria for program performance, existing standards of efficiency, and the necessary evidence. In addition, the evaluation process involves engaging stakeholders who serve as contributors offering their unique perspectives. They play a key role in selecting the appropriate standards as the lack of their involvement can easily result in assessment findings being ignored or criticized.
Another important phase is describing the problem, which affects what standards are applied regarding evaluating the outcomes. Milstein and Wetterhall (2021) note that this description allows to “illustrate the program’s core components and elements, its ability to make changes, its stage of development, and how the program fits into the larger (…) environment” (para. 37). This stage implies an identification of the purpose, which translates into possible criteria and standards applied by evaluators. Four general purposes are “to gain insight, (…) to improve how things get done, (…) to determine what the effects of the program are, (…) to affect those who participate in it” (Milstein & Wetterhall, 2021, para. 54). For example, if the purpose of the project is to improve how things get done, then the standards should emphasize program outcomes. Possible standards may include sustainability, reach, feasibility, and fidelity.
Determining Evaluation Standards
It is evident that despite a variety of existing standards that can be applied to any program, evaluators have to make case-by-case decisions and utilize criteria suitable for assessing a specific program. To determine which standards to apply, they have to go through multiple stages of analysis. Firstly, as mentioned earlier, they have to determine the purpose of the project. Secondly, they must identify the group or groups that will employ the evaluation findings (Centers for Disease Control and Prevention (CDC), 2012). For instance, if local authorities are the primary stakeholders who will utilize evaluation results to justify resource allocation for a specific project, then the standard of utility and the outcome of reach are going to be prioritized above feasibility, propriety, or accuracy. Thus, evaluators also have to gain a clear understanding of how the evaluation findings will be used (Hampton, 2021). For example, the results may help to mobilize community support, justify resource allocation, document effectiveness, or solicit more funding.
The aforementioned actions allow evaluators to determine whether the standard of utility should be applied. However, when it comes to other standards, evaluators must ask themselves different questions. To determine whether to utilize feasibility, experts need to consider the project’s developmental phase, intensity, and the measurability of the program’s components (CDC, 2012). When it comes to intensity, a great example might be that simple programs, even if proven to be extremely useful, cannot “be expected to make significant contributions to distal outcomes of a larger program, even when they are fully operational” (CDC, 2012, para. 20). As for propriety, the decision to apply this standard depends on the estimated ability of the program design to detect any unintended outcomes (CDC, 2012). Finally, regarding the accuracy, evaluators have to determine whether the focus and design of the project are broad enough to recognize failure and success and respond to stakeholders’ needs.
Conclusion
Evaluation is a multi-level, complex, yet effective strategy to distinguish effective projects from those, which do not justify the resources needed for them. It is the only method to demonstrate the results of investments accurately. In addition, the evaluation process allows us to improve existing practices and adopt resilient strategies worth the resources needed to implement them. Through the process of systematically gathering empirical and contextual data, evaluators can assess the significance of a specific project. However, to do that, they have to determine a set of applicable standards and criteria for further assessment. After all, evaluation should be a case-by-case process despite there possibly being a singular general framework with regulations and recommendations. It starts with identifying the object of evaluation, performance criteria, and standards, as well as the evidence needed. The stages that follow include engaging stakeholders and describing the program. Based on this analysis, evaluators can determine whether certain standards should be applied to a specific project.
Thus, the steps for deciding which evaluation standards to utilize include defining the purpose and stakeholders of the program, determining the use and users of the evaluation findings, as well components of the personal logic model used by an evaluator. In addition, it is crucial to take into consideration the project’s stage of development, intensity, and requirement resources. Furthermore, evaluators have to review evaluation questions and alternatives for the evaluation design with primary stakeholders. Lastly, they must consider the evaluation aspects of specific program components in their focus, including effectiveness, efficiency, and performance.
References
Centers for Disease Control and Prevention. (2012). Step 3: Focus the evaluation design. CDC. Web.
Dey, P. K. (2006). Integrated project evaluation and selection using multiple-attribute decision-making technique. International Journal of Production Economics, 103(1), 90–103.
Hampton, C. (2021). Chapter 36: Section 5. Developing an evaluation plan. Community ToolBox. Web.
Hogan, L. R. (2007). The historical development of program evaluation: Exploring the past and the present. Online Journal of Workforce Education and Development, 2(4), 1–14. Web.
Milstein, B., & Wetterhall, S. (2021). Chapter 36: Section 1. A framework for program evaluation. Community ToolBox. Web.
Picciotto, R. (2011). The logic of evaluation professionalism. Evaluation, 17(2), 165–180.
Schultes, M.-T., Kollmayer, M., Mejeh, M., & Spiel, C. (2018). Attitudes toward evaluation: An exploratory study of students’ and stakeholders’ social representations. Evaluation and Program Planning, 70, 44–50.