Introduction
The evaluation of the program is an important procedure in order to conclude about its effectiveness to be implemented in the concrete institution to address the set goals. The program that is selected for the evaluation is Cooperative Integrated Reading and Composition (CIRC) that is oriented to students studying in grades from 2 to 6. This program was developed by Nancy Madden and Robert Slavin in the 1980s, and it became actively integrated into the US K-12 institutions in the 1990s (U.S. Department of Education, 2012).
The CIRC program was revised several times, and the developers also proposed the Spanish variant of the program. The program is discussed by researchers as increasing the students’ results in reading, writing, and comprehension significantly, and it is important to describe the program and its components in detail.
Description of CIRC Components
The CIRC program was developed in the context of the Success for All educational model. The main components of the program are the focus on the tasks and direct instructions related to the certain story discussed in the classroom; the focus on the readÂing comprehension; and the focus on writing and composition activities based on the previous tasks. The lessons last 90 minutes, and students work in pairs and teams. The focus is on the work of a cooperative team within which students read, retell, and discuss stories, practice in answering the questions regarding stories, and write short papers (U.S. Department of Education, 2012).
Much attention is paid to checking the works of partners and testing (Mustafa & Samad, 2015). The work in the team is an important element of the program, and students are assessed not only regarding their successes in reading and writing but also regarding their cooperation in the group.
CIRC Purpose, Goals, and Objectives
The purpose of the CIRC program is to help students in developing their reading, comprehension, and writing skills while working in the cooperative environments. The goal of the program is to contribute to developing students’ skills in the fluent reading, easy understanding of texts, writing, editing, and proofreading texts associated with the learned material. As a result of CIRC activities, students learn to read without mistakes, pronounce words correctly, and enhance their vocabulary (U.S. Department of Education, 2012).
They also develop skills in summarizing texts, answering the questions on the text content, and writing short papers without spelling mistakes (Zainuddin, 2015). In this context, the purpose of the CIRC program is to use the advantages of the cooperative learning in order to develop students’ skills in reading, comprehension, and expressing their thoughts in the writing.
Literature Review
In spite of the fact that the CIRC program is implemented in public and private K-12 institutions of the United States during more than three decades, the research on the program’s effectiveness is rather limited. According to the report provided by the U.S. Department of Education (2012), the majority of studies oriented to examining the program’s outcomes were conducted in the 1990s. The articles were focused on determining the effects of the program implementation on students’ reading, writing, and comprehension in different class settings and environments. The studies were conducted in Ohio, Pennsylvania, and Iowa with the overall sample population of more 800 students.
It was found that students’ skills related to alphabetic, reading fluency, story retelling, and comprehension improved after the program implementation while comparing the results with the control groups of students (U.S. Department of Education, 2012). The researchers concluded about the high potential of the program for the development of students’ skills in reading and supportive writing. Students demonstrated the improvements regarding the understanding of words, skills in spelling, and different types of reading, including partner, independent, and group reading (U.S. Department of Education, 2012). In the 2000s, the studies focused on the CIRC program effects were continued, and today, the interest in the CIRC’s potential to improve the English learning increased significantly.
In his article, Durukan (2011) identified benefits of implementing the CIRC program in schools instead of traditional literacy programs and concentrated on the appropriateness of the instructions for beginning readers. Zainuddin (2015) found that the CIRC program is appropriate to develop skills in relation to reading and comprehending different types of texts, including descriptive ones. It is also noted in the literature that in comparison to Accelerated Reader that also was developed in the 1980s, the CIRC program can function as an independent course for reading in grades 2-6 when Accelerated Reader based on the software is usually integrated as an additional course to support the basic reading program in K-12 institutions (U.S. Department of Education, 2010a). These conclusions are also supported by findings of Karafkan and Aghazadeh (2015).
The CIRC program is also discussed as more effective to affect the students’ reading and writing in comparison to Peer-Assisted Learning Strategies because of its focus on the variety of reading instructions and types of working with partners (U.S. Department of Education, 2010b). According to Gupta and Ahuja (2014), the CIRC program effects on reading fluency are reported as significantly higher, and the researchers demonstrate the possibility of implementing the program not only in grades 2-6 but also in the seventh grade with noticeable advantages for students.
The CIRC program is discussed as an effective alternative to other reading programs because it was developed decades ago, and it was modified several times according to the recommendations of previous evaluators (Biancarosa & Cummings, 2015; Lovett, Lacerenza, Steinbach, & De Palma, 2014). According to Hadiwinarto and Novianti (2015), it is possible to identify two meanings that provide the background for the CIRC model that are “the cohesiveness of cooperative groups between students and the cohesiveness of reading and writing skills” (p. 118).
Referring to the program statement, it is important to note that “the improvement of reading skill through the CIRC learning model is also followed by the improvement of writing skill” (Hadiwinarto & Novianti, 2015, p. 118). These principles allow the simultaneous development of skills in students in the context of pair and group activities that are appropriate for their age (Begeny, Ross, Greene, Mitchell, & Whitehouse, 2012; Christ, Zopluoglu, Monaghen, & Van Norman, 2013; Hadiwinarto & Novianti, 2015).
Following Mustafa and Samad (2015), the CIRC program is effective “to assist students in reading and writing because it trains them to summarize, identify main ideas, answer literal questions, predict and clarify, and make inferences,” and students are “expected to have sufficient vocabulary to generate ideas to produce a text and also comprehend the kinds of text to be written” (p. 32). This approach allows the complex development of reading, retelling, comprehension, and writing skills in their cohesiveness that is also not observed in other programs.
Purpose of Evaluation
The reason to select the CIRC program for the evaluation is that this curriculum is one of the most actively integrated courses in K-12 institutions since the 1990s, and it is important to compare it to the other programs popular in the US schools. In spite of the proved effectiveness of the program to develop the students’ skills in reading, writing, and expanding the vocabulary, there are many other alternative programs that have the high potential for improving the students’ reading skills (U.S. Department of Education, 2012). The improvement of students’ reading and writing skills is the priority for school authorities in states, and each school district aims at selecting the most effective program to be implemented in different grades (Lovett et al., 2014). The availability of different programs allows choosing the most effective curricula that can be successfully integrated into other settings and which potential was evaluated by experts in the field of education.
The stakeholders of the CIRC program are school district authorities, directors of elementary and middle schools, and teachers of courses in reading and writing in classrooms with the high number of students with the limited literacy skills. Although many researchers evaluated the CIRC program in the context of the US schools, the studies and evidence-based conclusions are related to the period of the 1990s (Durukan, 2011; U.S. Department of Education, 2012).
The number of recent studies to prove the effectiveness of this program in comparison to the latest ones is rather limited. The most recent studies are related to the implementation of the program in the countries other than the United States and classes for English learners (Gupta & Ahuja, 2014; Mustafa & Samad, 2015). Nowadays, school authorities and teachers working in grades 2-6 are interested in results of the current evaluation of the CIRC program to conclude whether it is appropriate to be integrated into modern class settings with the high level of effectiveness that was noted during the previous studies and evaluations.
While referring to the stakeholders’ interest in evaluating the CIRC program, it is important to state that the purpose of the evaluation is to assess the effectiveness of the program for working with students from grades 2-6 and improving their skills in reading, text comprehension, and writing. The evaluation project can contribute to the discussion of the CIRC program’s strengths and weaknesses significantly as it is possible to concentrate on the areas that were not analyzed by evaluators previously. Any additional research in the field provides the opportunity for the high-quality assessment of the program and strategies used to achieve the program’s goals.
Evaluation Plan
Selection of the Evaluation Model
The model that should be selected for the evaluation of the CIRC program is known as the objectives-oriented approach. The reason to focus on this model is that it allows evaluating the program while concentrating on the concrete objectives that were set by the program developers. As a result, it is possible to measure the program achievements referring to real outcomes and conclude about the program’s effectiveness (Grammatikopoulos, 2012; Wiesenberg, 2013).
It is important to note that the program should be developed effectively in order to achieve the stated goals and objectives, and it is the main factor according to which educators choose the implementation of this program in an institution. The objectives-oriented approach directly addresses these goals of the evaluation. Currently, there are many alternative programs adopted in K-12 institutions in order to improve students’ skills in reading, comprehension, and writing (U.S. Department of Education, 2010a; U.S. Department of Education, 2010b).
Educators need to choose such program among the proposed ones that can achieve the objectives directly and effectively. The evaluation of the CIRC program with the help of the objectives-oriented model allows determining the extent to which the set reading and writing objectives are met (McMahon & Cullinan, 2014; Reutzel, Child, Jones, & Clark, 2014). The model focuses on evaluating outcomes in order to provide the adequate assessment of the program, and this simple method is often used by evaluators because educators prefer to implement efficient programs that can guarantee the achievement of curriculum goals.
Evaluation Questions
The objectives-oriented evaluation is guided by those goals and aims that were formulated by the developers of the CIRC program. The evaluator focuses on assessing the program in terms of the concrete outcomes that can be expected after its implementation. The first objective is to develop students’ readÂing skills to the extent that allows the easy comprehension of the text. The other objectives related to reading include the following ones: to develop skills in summarizing and retelling stories based on the improved comprehension, to develop the students’ vocabulary, and to develop students’ writing skills while responding to the text-based questions (U.S. Department of Education, 2012).
These concrete objectives provide the background for the formulation of questions used for the evaluation:
- Did the CIRC program achieve the set objectives?
- To what extent did the CIRC program achieve the set objectives?
- Did the scores of student taught according to the CIRC program differ significantly from the scores of students taught according to the traditional literacy program?
Sampling Technique
The evaluation of the CIRC program is conducted in a K-12 institution of the school district in Texas. In order to conduct the evaluation of achieving the CIRC program objectives in all grades where the course is implemented, it is necessary to use the cluster random sampling technique and form two groups of students for each grade, including the test group and the control group (Frye & Hemmer, 2012; Tipton, Hedges, Vaden-Kiernan, Borman, & Sullivan, 2014).
For this purpose, students from grades 2-6 where the CIRC program is implemented should be randomly selected according to the principles of the cluster random sampling in order to compare their results with those students who study in grades 2-6, where other literary programs are followed. The focus on test and control groups from each grade allows comparing the actual program outcomes and concluding regarding the changes in students’ performance and achievements of the set objectives. The number of students in a test and control group for each grade is ten persons. The final sample is rather small and includes 50 students from the test group.
Data Collection Technique
The CIRC program objectives determine what type of information should be collected for the further evaluation. It is important to collect both the qualitative and quantitative data in order to guarantee the complex evaluation of the program. The objectives-oriented model does not limit evaluators in terms of data collection tools (McMahon & Cullinan, 2014; Wiesenberg, 2013). The qualitative data related to the 2014-2015 academic year, such as the samples of students’ written works and notes regarding student’s retelling, are collected for the further qualitative assessment.
The quantitative data include scores in reading and comprehension set according to the state standards in Texas. The quantitative data were collected with the help of standardized reading and writing achievement tests used in the Texas K-12 institutions in the 2014-2015 academic year in order to assess the results of students in different grades (Hung, Hsu, & Rice, 2012). For the purpose of this evaluation project, the data related to students from the sample were retrieved with the help of grade teachers and administrators for the further analysis.
Data Analysis
The collected data were evaluated with references to the results of the data analysis. The qualitative data were assessed according to scales used for the assessment of students’ achievements in writing as the part of the work with texts and in text retelling. The scales were developed by teachers working in grades 2-6 according to the standards of performance adopted in Texas regarding composition writing and retelling.
The scores were analyzed with the help of summing the number of right points and answers set following the used scales (Durukan, 2011; West, 2015). According to the results of the qualitative data analysis, it is possible to note that students taught according to the CIRC program in grades 2-4 had the scores higher by more than 10% in comparison to the scores demonstrated by students from control groups. For students from grades 5-6, the difference between scores is more than 5% (Appendix A).
The quantitative data on test results were analyzed with the help of such quantitative data analysis tool as the t-test. The use of the t-test is appropriate for the evaluation because it allows determining the possible significant difference between two sets of the data (Hobson & Burkhardt, 2012; Mustafa & Samad, 2015). For the purpose of this project, the compared sets of data are scores demonstrated by students taught according to the CIRC program and students taught according to the other literacy program.
The results indicate that the statistically significant difference is observed in terms of scores typical of students participating in the CIRC program and students from the control group for grades 2-4, and there is no statistical difference for grades 5-6 (Appendix B). It is possible to speak about the program effectiveness in terms of being used in grades 2-4 to develop students’ skills in reading and composition, but its effectiveness for older students is not supported.
Ethical Issues
The selection of data collection tools is associated with such ethical issues as the bias. The teachers working in grades 2-6 during the 2014-2015 academic year graded the students’ works, tasks completion, and tests independently, without the involvement of the evaluator. As a result, there is a risk that results can be influenced by the teachers’ bias in relation to students performing well or poorly. The other ethical issue is associated with the adherence to the Program Evaluation Standards and the focus on rights of human subjects (Yarbrough, Shulha, Hopson, & Caruthers, 2010). While conducting the evaluation, it is important to respect the rights and interests of subjects, and this principle guided the cooperation with teachers during the evaluation.
Recommendations
The evaluation results indicate that there is a statistically significant difference between the reading and writing outcomes of the CIRC program and traditional literacy program for grades 2-4, but there is no difference for students’ scores and achievements in grades 5-6. The changes in reading determined with the help of scales are also more important in relation to the students studying in grades 2-4.
These results demonstrate the necessity of improving the CIRC program approach in order to develop the reading, comprehension, composition, and writing skills of students in grades 5-6. The recommendations to implement include modifying the instructions to address the students’ needs and level in reading, paying more attention to the composition and writing component of the program, and focusing on the element of the cooperative learning as effective for the determined age group of students.
The recommendations should be implemented following the developed strategy. The first step is to modify the instructions associated with reading in pairs and groups in order to draw the students’ attention to these activities (Gupta & Ahuja, 2014). The second step is to expand the number and types of story-related activities used for students in grades 5-6 in comparison to the activities used in grades 2-4 (Mustafa & Samad, 2015).
The third step is to improve the instructions related to composition and writing and guarantee the integration of the CIRC program with other components of the language arts (Hadiwinarto & Novianti, 2015). The final step is to focus more on the cooperative element of the program and foster the pair and group collaboration of learners. This strategy covers the areas that were noted as requiring the improvement with references to the evaluation results.
The main stakeholders interested in the results of the CIRC program evaluation are school district administrators, teachers, parents, and the wide community of educators. The plan to communicate the results of the evaluation and proposed recommendations includes such actions as the preparation of the brief report on findings for administrators of the school district where the evaluation of the program was conducted. The next step is the preparation of the PowerPoint presentation to provide such stakeholders as teachers and the community with the actual results of the evaluation in the form appealing to the public. The other possible step is writing an article on the findings of the evaluation to be published in the scholarly journal.
Conclusion
It is important to answer the evaluation questions set before starting the project according to the indicated findings. The CIRC program achieved the set objectives in relation to developing reading and composition skills of students, but its benefits are more evident for beginning readers from grades 2-4. The extent to which the CIRC program achieved the set objectives is medium for students from grades 5-6, and it is large for students from grades 2-4.
The scores of student taught according to the CIRC program differ significantly from the scores of students taught according to the traditional literacy programs only in the case for beginning readers, and the results are rather insignificant for learners in grades 5-6. The findings allow assessing the CIRC program as effective to be applied in the elementary school, and it can be referred to as the alternative program or additional instruction in grades 5-6.
References
Begeny, J. C., Ross, S. G., Greene, D. J., Mitchell, R. C., & Whitehouse, M. H. (2012). Effects of the Helping Early Literacy with Practice Strategies (HELPS) reading fluency program with Latino English language learners: A preliminary evaluation. Journal of Behavioral Education, 21(2), 134-149.
Biancarosa, G., & Cummings, K. D. (2015). New metrics, measures, and uses for fluency data: An introduction to a special issue on the assessment of reading fluency. Reading and Writing, 28(1), 1-7.
Christ, T. J., Zopluoglu, C., Monaghen, B. D., & Van Norman, E. R. (2013). Curriculum-based measurement of oral reading: Multi-study evaluation of schedule, duration, and dataset quality on progress monitoring outcomes. Journal of School Psychology, 51(1), 19-57.
Durukan, E. (2011). Effects of cooperative integrated reading and composition (CIRC) technique on reading-writing skills. Educational Research and Reviews, 6(1), 102-109.
Frye, A. W., & Hemmer, P. A. (2012). Program evaluation models and related theories: AMEE Guide No. 67. Medical Teacher, 34(5), e288-e299.
Grammatikopoulos, V. (2012). Integrating program theory and systems-based procedures in program evaluation: A dynamic approach to evaluate educational programs. Educational Research and Evaluation, 18(1), 53-64.
Gupta, M., & Ahuja, J. (2014). Cooperative Integrated Reading Composition (CIRC): Impact on reading comprehension achievement in English among seventh graders. International Journal of Research in Humanities, Arts and Literature, 2(5), 37-46.
Hadiwinarto, H., & Novianti, N. (2015). The effects of using CIRC model on the English learning skills among junior high school students. Journal of Education and Learning, 9(2), 117-124.
Hobson, K. A., & Burkhardt, J. T. (2012). A lesson in carefully managing resources: A case study from an evaluation of a music education program. Journal of MultiDisciplinary Evaluation, 8(19), 8-14.
Hung, J. L., Hsu, Y. C., & Rice, K. (2012). Integrating data mining in program evaluation of K-12 online education. Educational Technology & Society, 15(3), 27-41.
Karafkan, M. A., & Aghazadeh, Z. (2015). Investigating the effects of Group Investigation (GI) and Cooperative Integrated Reading and Comprehension (CIRC) as the cooperative learning techniques on learner’s reading comprehension. International Journal of Applied Linguistics and English Literature, 4(6), 8-15.
Lovett, M. W., Lacerenza, L., Steinbach, K. A., & De Palma, M. (2014). Development and evaluation of a research-based intervention program for children and adolescents with reading disabilities. Perspectives on Language and Literacy, 40(3), 21-32.
McMahon, J., & Cullinan, V. (2014). Education programmes for young children with autism spectrum disorder: An evaluation framework. Research in Developmental Disabilities, 35(12), 3689-3697.
Mustafa, F., & Samad, N. (2015). Cooperative Integrated Reading and Composition technique for improving content and organization in writing. Studies in English Language and Education, 2(1), 31-49.
Reutzel, D. R., Child, A., Jones, C. D., & Clark, S. K. (2014). Explicit instruction in core reading programs. The Elementary School Journal, 114(3), 406-430.
Tipton, E., Hedges, L., Vaden-Kiernan, M., Borman, G., & Sullivan, K. (2014). Sample selection in randomized experiments: A new method using propensity score stratified sampling. Journal of Research on Educational Effectiveness, 7(1), 114-135.
U.S. Department of Education. (2010a). What Works Clearinghouse: Accelerated Reader. Web.
U.S. Department of Education. (2010b). What Works Clearinghouse: Peer-Assisted Learning Strategies. Web.
U.S. Department of Education. (2012). What Works Clearinghouse: Beginning reading intervention report, Cooperative Integrated Reading and Composition. Web.
West, S. E. (2015). Evaluation, or just data collection? An exploration of the evaluation practice of selected UK environmental educators. The Journal of Environmental Education, 46(1), 41-55.
Wiesenberg, F. (2013). A critical appraisal model of program evaluation in adult continuing education. Canadian Journal of University Continuing Education, 26(2), 79-109.
Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2010). The Program Evaluation Standards: A guide for evaluators and evaluation users. New York, NY: Sage Publications.
Zainuddin, Z. (2015). The effect of cooperative integrated reading and composition technique on students’ reading descriptive text achievement. English Language Teaching, 8(5), 11-21.