The rationale for doing the evaluation design
The rationale for designing the framework for Waste Diversion Program in Ontario, Canada is to improve waste management efforts through diversion activities. This is vital for the program because the framework will allow the evaluator to determine what the Waste Diversion Program has achieved in Ontario within a specified period. Moreover, data obtained will help in understanding and describing the program processes. It would help key stakeholders to develop the best methods to improve solid waste management in Ontario, as well as improve the entire program.
The major aim of this framework is to ensure that Ontario enhances quality, effectiveness and efficiency in the Waste Diversion Program. Therefore, decision-makers (local government and other private waste management bodies) in the Waste Diversion Program in Ontario will not make any uninformed decisions about program evaluation processes. The framework will provide all the necessary steps.
The program focuses on Municipal Solid Waste (MSW). These are mainly trash or garbage, which come from residential areas, factories, institutions, and commercial entities among others. Such waste materials could be paint, newspapers, glass materials, plastic bottles, and e-waste materials.
Incinerators, landfills, and other traditional methods of managing waste materials are no longer effective as Ontario City generates huge amounts of wastes. Consequently, it is hard pressed to look for alternative ways of managing waste materials, particularly solid wastes and other hazardous wastes. This calls for effective disposal of wastes through planning and controlling them in economical and environmentally friendly manner.
Therefore, Ontario opted for a Waste Diversion Program. Waste diversion aims to reduce the volume of wastes for landfill.
By the early 1980s, Ontario had already started to grapple with a crisis of managing solid wastes due to limited capacities of landfill. This resulted into initiatives to promote recycling through business and community efforts. Such strategies focused on changing the laws and regulations on soft drinks.
Ontario focuses on industrial, commercial, and institutional domains to facilitate waste diversion. These are the major waste generators in Ontario. These domains or divisions have significant responsibilities in managing and diverting wastes in Ontario. They rely on the 3Rs (Reduction, Reuse, and Recycling) strategies to reduce the amount of solid waste materials that go into landfills.
In addition, there have been several initiatives to encourage waste diversion through regulatory frameworks, technical and financial aid, public education, and manufacturing of recyclable materials. Therefore, Ontario focused on industrial, commercial, and institutional sectors to enhance waste management plans, keep data, and audit their waste management activities. These are fundamental data, which help the government to understand waste dynamics in Ontario, develop policies, and allocate resources based on them.
In 2010, Ontario disposed only 9,247,415 tones of wastes in landfills and diverted a small percentage, which declined by six percent relative to the year 2008 (Statistics Canada, 2013).
A logic mode of the Waste Diversion Program
The logic model indicates several aspects of the cause-effect relationship involved in waste management and diversion efforts. These are resources and activities, which are responsible for bringing about the presumed waste diversion efforts and the success of the Waste Diversion Program and management in Ontario.
The logic model would allow the local government and other stakeholders to improve on the program and focus on it. The logic model would show assumptions about the Waste Diversion Program effectiveness and offer a frame of reference for evaluators as they strive to account for all variables (McDavid, Huse and Hawthorn, 2013). At the same time, details in the logic model shall act as a basis for assessing other aspects of the Waste Diversion Program, which might be difficult to measure directly. For instance, it may be simple to evaluate the overall impacts of reduction in pollution from known sources if the interventions are effective and rely on prior understanding of effective programs.
This logic model would recognize technical aspects of waste management and diversion efforts due to their multiple relations and several stakeholders. Therefore, several diverse activities will be a part of the logic model. The evaluator will clarify several sources of information into residential and non-residential. This would provide a well-rounded account of the program. The evaluator would conduct a further discussion with different stakeholders to establish the accuracy of the Waste Diversion Program in Ontario. While it will be imperative to observe current activities on waste diversion, the evaluator would review historical data to establish changes in the Waste Diversion Program within a reasonable period. The program evaluation shall focus on other factors, such as job creation, political efforts, local and private investments, community participation and other variables with significant impacts on the program outcomes.
The research design
The program design would facilitate effective uses of available resources while addressing issues of critical interests to program stakeholders.
The evaluator would use a scientific research method to assess the Waste Diversion Program in Ontario. The method would ensure that only reliable data are collected to address stakeholders’ concerns i.e., the selected method would account for primary users of the report, its use and questions. On this note, a longitudinal study method would be adopted by using several historical data to set up a comparison between selected periods. This form of observational study shall be simple to implement and compare changes over the selected periods. It would involve a retrospective study in which the evaluator would go back in waste management records to look for trends since the inception of the Waste Diversity Program. The method would not manipulate any variables.
The method will offer an effective approach of comparing variables over specified time. However, the nature of the study might not allow for any randomization. Thus, there is no freedom to choose subjects for the study. At the same time, it would be difficult to determine any other confounding variables, which could affect waste management program in Ontario within the periods chosen. Therefore, a lack of randomization may confound study results. To account for such weaknesses in the design, the evaluator would consider several aspects.
First, the evaluator will define the study purpose clearly. This would eliminate any possible haste in decision-making on how the program evaluation should run. The main purpose of this program evaluation would be to provide insights that may be used to change waste diversion practices in Ontario. Moreover, it would show achievements and challenges over specified periods. Therefore, the evaluator shall use results to describe waste diversion processes, enhance waste diversion activities and improve the overall program strategies. The evaluation results will facilitate quality improvement, program effectiveness and efficiency.
Second, the evaluator will define evaluation questions to mark the limit for the study by clarifying specific aspects of the Waste Diversion Program that shall be addressed. Developing specific evaluation questions will allow various stakeholders to identify critical areas of the program, which they consider important and require responses. The process would involve negotiation and prioritization of questions among various stakeholders to ensure that the design process accounts for critical elements. This process may expose diverse views of stakeholders.
Third, the design process will account for users of the evaluation report. Design processes could have direct effects on users of the evaluation report. Stakeholders’ involvement would ensure that the evaluator understands the intended use of the evaluation report, prioritize questions and avoid any possible bias or irrelevant contents.
Fourth, the evaluation design will account for uses of the results. This refers to specifics ways in which data collected and the generated report would help in improving the program. The evaluator shall avoid vague terms to enhance usability of results. This might improve chances of addressing specific needs of the program. The evaluator would plan uses of the report based on the intended and prioritized needs of stakeholder while taking into account developments in waste diversion efforts in Ontario. All uses of the report could reflect diverse nature of stakeholders in waste management.
The measures to operationalize the main constructs in the evaluation of the program
The evaluator will use existing data from Statistics Canada’s database on waste management industry surveys for the specified periods (see the attached sample of data source for evaluation). These are data, which focus on both government and private sectors in waste management across different cities in Canada. This implies that study variables are already defined.
Some the indicators would include:
- Wastes disposed
- Waste per capita
- Tones of wastes diverted
- Changes in percentages in the rates of diversion
- Capital expenditures and revenues
- Types of wastes disposed
- Waste management methods
- The number of businesses and employees
These are variables, which highlight changes in waste management activities and could be important in explaining such activities over a long period to determine the effectiveness of the Waste Diversion Program in Ontario.
It would be important to evaluate the validity and reliability of the existing data. Collected data need to be reliable and valid to present expected outcomes. Determining reliability of such data involves conducting an assessment to establish reliability of the study data. The aim is to ensure that data collected are consistent with what they are designed to measure. Moreover, the evaluator must follow correct methodical procedures in testing and evaluating to make outcomes useful for the program improvement.
The evaluator shall establish the validity of the data by reviewing it with a panel of experts on waste management and diversion activities and performing some few tests. The process will determine the validity of the collected data. In other words, data gathered for evaluation must reflect aspects of waste management and diversion. They must also represent the study content, be suitable for various participants and comprehensive enough to answer all program issues.
Relaibility and validity of the collected data may provide several advantages to evaluators. For instance, data gathered should reflect ease of use through their highly structured patterns. This offers simplicity for evaluators during data quantification. The evalautor would retrieve relevant data and perform analysis with computer-based programs, which shall provide simple analysis processes and quick results with minimal errors. The strategy would rely on a taxonomic approach to enhance a comparison between periods. Overall, the evaluator will seek to prove that all collected data are consistent in the entire study periods. Consistency must account for different aspects of waste management and diversion. This would ensure that data analysis provides reliable and valid results.
The evaluator would use the program logic model to develop evaluation indicators or variables. The logic model would define the scope of indicators based on the identified program activities to expected long-term outcomes. It is imperative to align both qualitative and quantitative variables with the logic model, question concepts, available data and intended uses for the evaluation report. This will allow the evaluator to notice any changes in the performance of the Waste Diversion Program.
The main source of data would be public data on waste management from Statistics Canada. The evaluator believes that Statistics Canada has the most relevant public data on waste management activities across different areas in Canada. While the source may be adequate for the program evaluation, the evaluator may also consider other available sources to provide information for different indicators. Different sources of information shall offer an opportunity to include diverse views about the Waste Diversion Program. This would enhance the program credibility. Other sources of information for the study might include comments from the program team. Such comments could provide equally relevant ideas, which might not be within the existing database. Therefore, a combination of different perspectives will provide an inclusive outcome for the program. The evaluator will clearly state the criteria for determining data relevance so that stakeholders can provide their inputs.
Any statistical methods for interpreting the data gathered
Data analysis will involve descriptive analysis. The researcher shall identify all data suitable for tabulation and those for charts, graphs, frequency tables, percentage distribution and inferences.
The evaluator will use data analysis and synthesis to find and summarize the findings from the evaluated program. Data analysis shall ensure that all relevant patterns are identified and then synthesized to provide the overall picture of waste diversion activities in Ontario. The process of data analysis may involve working on separate key indicators and then bringing them together to provide patterns for inferences.
The evaluator will organize, compare, classify and highlight data so that the outcomes can provide facts. The overall outcomes must reflect the evaluation objective, questions, data types and other relevant information from different perspectives, which shall be a part of the study.
The evaluator would interpret the results in order to determine what findings means to stakeholders and waste diversion activities in Ontario. The evaluator will need adequate facts from historical data on waste management to draw an evaluative conclusion on the program. At the same time, the evaluator will assess evidence and interpret it to identify practical importance of the outcomes and learned elements from the program. Interpretation of evaluation outcomes shall require assessment and consideration of all facts and information drawn from all stakeholders.
Recommendations
The evaluator shall make recommendations based on the evaluation results. Therefore, stakeholders will understand what the program has achieved with specified resources.
Recommendations will consider organizational contexts so that the expected future outcomes match resource constraints and other potential issues during program improvement.
The costs of completing the evaluation
Summary
A program evaluation is an important method of determining if a program works or not. It acts as an effective tool for understanding performances of public initiatives, enhancing the existing Waste Diversion Program and other related waste management efforts. Moreover, program evaluation allows stakeholders to account for their investments through evaluation outcomes.
With the growing waste production and generation in Ontario, the program evaluation will show and draw attention on the common purpose of the local authority initiatives on waste management and demonstrate whether magnitudes of both public and private investments in waste management match the demand for such services.
The program evaluation framework relies on the best practices and standards in order to improve waste management practices. It offers a strong practical approach to the program evaluation methods. Hence, the outcomes would be reliable and valid.
This program framework shall guide the evaluation processes of the Waste Diversion Program in Ontario. Evaluators may also apply it in other settings because of its general principles and standard guidelines. Moreover, this evaluation framework will provide specific approaches to the Waste Diversion Program in Ontario because it shows relevant variables and further operationalize specific approaches to indicators.
Therefore, the recommended program evaluation framework is one that Ontario local authority and other stakeholders in waste management can rely on to enhance and account for their solid municipal waste management and waste diversion efforts.
References
McDavid, J. C., Huse, I., and Hawthorn, L. R. L. (2013). Program Evaluation and Performance Measurement: An Introduction to Practice (2nd ed.). Thousand Oaks: Sage Publications, Inc. Web.
Statistics Canada. (2013). Waste Management Industry Survey: Business and Government Sectors, 2010. Ottawa: Minister of Industry. Web.