Evaluation is an important part of the educational process. It is crucial for future success of the educational process and educational program. It shows what kind of work has been done, what objectives were accomplished and what are the benefits for participants. In education, evaluation is a basis for decision making that allows defining the alteration for the existing educational program.
There are approaches how to perform evaluation of the educational program, one of them is “program theory” which can also be represented in different ways. In this paper, we are going to provide a model of the future evaluation on an existing innovative program aimed at improving students’ proficiency in spoken and written English through writing English drama scripts and producing drama performance.
First of all, we should distinguish between evaluation, assessment and measurement and define what a program evaluation is, and what its benefits are. Thus, assessment and measurement are the components of evaluation. Assessment is a data-gathering and measurement “refers to the process by which the attributes or dimensions of some physical object are determined” (Kizlik 2011).
Assessment and evaluation can be distinguished according to three criteria: purposes, objects being assessed and steps in conduction. Thus, the purpose of assessment is to define the opportunities for learning and mane a diagnosis of the students’ performance, the assessed objects are students (class, group or course), the steps are taken in assessment are a number of targets (the results of instructions).
“Evaluation is more that a judgment of ultimate success or failure” (Frechtling 2007 p. 3). The purpose of evaluation is to improve the existing practices of educational program and provide results for future decision-making. As opposed to assessment, the objects being evaluated are particular programs, projects or few teachers that take part in projects.
Major steps taking when evaluating a program or a project are description of the program, providing a theory on how to input the outcomes and conducting a research on how to test the theory provided. According to Muraskin (1993), “evaluation can be an important tool in improving the quality of a program if it is integrated into the fabric of an educational program rather than added on after the fact.”(n. p.).
In this regard, evaluation can provide considerable benefits for the educational program. The program evaluation clearly articulates goals and objectives and strengthens the program design in such a way. Second, it serves a basis for decision making for improving the quality of the project. Third, it provides a contribution to making constructive changes which enhances the effectiveness of the project.
Next, it helps identify successes of the outcomes. Finally, it ensures connection between school program strategies and student outcomes (“Program Evaluation” 2009).
There are many reasons to conduct evaluation, among them are the possibilities to determine the effectiveness of programs, document of objectives that have been met, provide information about service useful to program staff and students, enable staff to provide changes to improve the program’s effectiveness (Muraskin 1993).
In order to create an appropriate strategy for program evaluation, it is necessary to know its logic. Weiss (1997) provides logic of evaluation analysis that consists of the steps that follow: describing program (input; activities; outcomes; causal links ),making comparisons (intended versus observed; pretest versus posttest; experimental versus control), developing and modeling the causal processes, locating unintended effects, examining deviant cases, ruling out alternative explanations, interpreting implications of findings, fashioning recommendations for changes and providing policy analysis that findings suggest.
This is the way the evaluation should be modeled and provided. Davidson (2005) suggests that “evaluation need to be designed and conducted in ways that address the information needs of program staff and other upstream stakeholders” (p. 23).
Evaluation of the program is usually implemented through the program theory. According to Patricia J. Rogers (2000):
Evaluations that are based on program theory have two essential elements: an explicit model of the program (in particular, the mechanism by which program activities is understood to contribute to the intended outcome and an evaluation which is guided by this model (p. 4).
There are minimum three component of the program theory: program activities, the intended outcomes, and mechanisms that lead to the intended outcomes. Program theory can be represented in different ways. There are four broad approaches that can be used to represent it: outcomes chain logic theory, pipeline logic models, realist matrices, and narratives.
We will focus on logic models that “show a sequence of results leading to the ultimate outcomes or impact of interest” (Funnell & Rogers 2011). In this paper, we will focus on logic modeling. So, according to Joy A. Frechtling, “a logic model is a tool that describes the theory of change underlying an intervention, product, or policy.
It characterizes the project through a system of elements that includes components and connections, with context being an important qualification” (p. 1). It helps define the necessary elements and areas of the project or program that needs evaluation and where evaluation will be crucial.
The basic elements of the logic model are inputs, activities, outputs and outcomes. In order to start a logic modeling, one should: explain concepts to participants, provide relevant materials, identify goals, outputs and activities, develop relationships and define whether the inputs are sufficient for accomplishment. (Alkin 2010).
In the process of evaluation of the educational program, logic modeling is important because it presents the mechanisms that lead to the improvement of the students’ academic performance and opens new strategies for the project improvement.
As Wholey, Hatry, & Newcomer (2004) state, “having logic modeling in place at the beginning of the evaluation is important because it serves as an advance organized or focusing mechanism for the evaluation and the measurement of key variables or performance indicators” (p. 96).
Thus, as it has already been mentioned, the project that under consideration is an innovative educational program aimed at improvement of the students’ proficiency in the English language. Future program evaluation of this program, which is based on the logic modeling, will help improve this innovative program.
In order to provide a logic model for evaluation, we should define goal and objectives of the program which are essential to know its goal and objectives to define resources and activities that constitute the components of the logic model. Thus, the goal of this program is “to enable students to write English drama scripts and produce drama performances” (“Project” 2008). The goal defines the objectives, with are:
To improve the proficiency in spoken and written English of the students, promote the appreciation and participation of both teachers and students in the performing arts, arose students’ interest and develop their creativity by producing a multi-arts performance, provide a means of staff development for the English Department and promote drama in the community by giving performances to primary schools in the district. (“Project” 2008).
“Evaluation is the systematic collection of information about activities, characteristics, and outcomes of projects to make judgments about the project, improve effectiveness, and/or inform decisions about future programming” (“Designing Evaluation” n. d.). To provide the logic model for this program, we should focus on these issues and define resources, activities, outputs and outcomes of the program.
The inputs of the program include the resources needed to run the program. These can be students and teacher staff that participate in the program, funding, partners, program materials and equipment needed for drama classes and performances.
The activities are the workshops, staff training, planning and developing of the program, coordination and network, data collection, writing classes and drama performances.
The outputs include all accomplishments of the activities mentioned earlier. These can be the organization of events, produced program materials, tests, etc.
Outcomes are positive results of the activities and benefits for students, such as the achievement of the goal and objectives of the program.
All components of the logic model influence each other, it can be demonstrated though the diagram that follow:
Program Logic Model
This program logic model involves different components of the program and helps identify key variables that should be included in it. It is based on the goals and objectives of the program and provides the mechanism of their implementation. It also indicates the activities that will lead to improvement of the innovative program.
Thus, evaluation is an important part of the educational process. It provides the basis for decision making and helps define strategies for educational project improvement. Logic modeling is one of the best ways of presentation of the program evaluation as it provides checks at different stages of the educational program.
Reference List
Alkin, M. C. (2010). Evaluation Essentials: From A to Z. New York: Guilford Press.
Chen, H. (2005). Practical Program Evaluation: Assessing and Improving Planning, Implementation, and Effectiveness. Calif: SAGE.
Davidson, E. J (2005). Evaluation Methodology Basics. Thousand Oaks, Calif: Sage.
Designing Evaluation. Office of Education and Sustainable Development NOAA. Web.
Frechtling, J.A. (2007). Logic Modeling Methods in Program Evaluation. Calif: Jossey-Bass.
Funnell, S. C., & Rogers, P. J. (2011). Purposeful Program Theory: Effective Use of Theories of Change and Logic Models. New York: John Wiley and Sons.
Kizlik, B. (2011). Measurement, Assessment, and Evaluation in Education. Web.
Muraskin, D. (1993). Understanding Evaluation: The Way to Better Prevention Programs. Web.
Program Evaluation. (2009) Web.
Project 2008/0637. Web.
Rogers, P. (2005). Program Theory: Not Whether Programs Work but How They Work.
Stern, E. (2005). Evaluation Research Methods. Vol III. Thousand Oaks, Calif: Sage. pp. 3 – 30.
Weiss, C.H. (1997). Evaluation. 2nd Ed. NJ: Prentice-Hall.
Wholey, J. S., Hatry, H. P., & Newcomer, K. E. (2004). Handbook of Practical Program Evaluation. New York: John Wiley and Sons.