Introduction
This paper will identify the design of the study used in gathering information for the paper titled “The Effect of Accuracy of Instruction, Teacher Delivery and Student Attentiveness on Musicians’ Evaluation of Teacher Effectiveness.” The paper will discuss the issues in quantitative design such as validity, reliability and generalizability, and application of statistical tests. Additionally, the paper will identify the unanticipated results in comparison to the findings of previous research, and how the design used in the study might have been strengthened through alternative procedures.
Validity
Validity of research data is based on an evaluation of whether a particular research is really measuring what it is supposed to be measuring (Thyer, 2001). There are two types of validity as discussed below.
Internal validity
Internal validity refers to the approximate truth about suppositions derived concerning cause and effect in a given research work (Thyer, 2001). Internal validity is usually relevant in research works that try to ascertain a causal relationship. The key question is whether the changes observed in a particular research program can be attributed to the research methods used or to other causes (Thyer, 2001).
In the research article being investigated in this paper, the researcher used attributes such as eye contact, vocal delivery and body language to evaluate their performance. While these attributes can be relied on at particular time, they are bound to change depending on the prevailing conditions. An inference can be made that the students’ expressions at the time of study were caused by factors such as the tone used in the research. Body and feelings are not measurable and change from time to time and thus affect the validity of the data unless the research is done time and again.
The researcher used a sample (a representation of the entire population) of 168 students divided into four categories each with 42 participants. To increase internal validity, all the four categories could have participated in all the activities that were tested. This would have increased the representation of the sample in the tested areas.
External validity
External validity refers to a concept of making generalizations. In particular, it refers to the rough estimate of truth and inferences that lead to particular conclusions on findings in a given research (Denzin & Lincoln, 2005). In essence, it means applying the findings that relate to a particular sample to a larger population (Cohen, Manion & Morrison, 2003). Making generalizations implies that the results obtained should be very common to the sample in order to facilitate application of the same findings to a wider area.
The findings in the article under evaluation suggest that a high or low delivery teacher service may influence the students’ secondary perceptions of music and affect effective teaching of the subject rather than the level of accuracy of the teachers’ instruction and the students’ social behaviors. These results are similar to the findings obtained by Weeks (1991) and the author therefore suggested that they can be applied to a wider population. But the results also contradict the findings tabled by Young and Shaw (1999) that some college students give some teachers higher ratings if the teachers score low in certain components of the 25-item course evaluation form for teachers. Hence, there was need to research why the students’ perceptions of teachers were varied based on the 25-item teacher evaluation form. This could have provided more conclusive results about students’ behavior.
Using the findings of the research, the researcher tried to generalize the behavior of students in Music in many schools. However, the truth of the generalization may be insignificant because behavior usually changes with reference to the prevailing environmental and attitudinal conditions (Cohen, Manion & Morrison, 2003). There is therefore scanty evidence that the results obtained in the research could hold if the same research was carried out in another school. An alternative method for the researcher to obtain more reliable statistics would have been to use a larger sample consistently in all the tested categories.
The threats to external validity are summarized as being caused by people, places or times (Thyer, 2001). As earlier mentioned, it is not possible to generalize different individuals’ behavior to a population since people have different attitudes and opinions. Additionally, prevailing conditions such as place and time affect people’s reactions at different times.
Reliability
Research reliability relates to the applicability of results to a group other than that used in a particular study (Denzin & Lincoln, 2005). Reliability is concerned with ensuring that the same method of gathering information can produce consistent results if it is used subsequent times. Thus if similar results are produced the methods of collecting data are said to be reliable (Cohen, Manion & Morrison, 2003).
In the research under evaluation, reliability of the research lies in the fact that a two-way analysis of variance (ANOVA) with repeated measures was used. The two-way ANOVA involves analysis of variance between two independent variables (Denzin & Lincoln, 2005) and this means that the results obtained gave a significant picture of the characteristics of the student population. Nevertheless, this was only an analysis and does not confirm that the research data are reliable since no-repeat sampling was done. But the author attempted to portray reliability by evaluating various attributes of students such as singing quality, verbal academic information, verbal feedback and so forth.
Although the author has used several attributes, reliability relies not on one exercise of research but the average of several repeat exercises. Therefore, the presented results cannot be relied on as the general the conclusive findings, there is need to carry out more similar exercises. In order to derive more convincing results, the researcher could have used more treatment groups. Treatment groups are the groups formed by making as many combinations as possible of the independent variables being analyzed (Creswell, 2008).
Generalizability
Generalizability involves making predictions about an event based on recurring experience in research exercises (Denzin & Lincoln, 2005). This is based on the dictum that if something occurs frequently, then it can be anticipated to occur in future (Cohen, Manion & Morrison, 2003). This is the essence of making hypotheses since researchers usually anticipate some kind of result. But the fact that hypotheses are based on probability implies that not all anticipated results are conclusive or exhaustive.
In the current evaluation, the author made some conclusions based on analysis of statistics by use of methods such as percentages. However, these analyses do not guarantee the accuracy of research findings since they are based on estimates. In addition, in order for generalizability to be applied, the research had to be done again and again. Therefore, while the researcher tried to compare the finding with past findings, there is still a difficulty in making the results generalizable since the different research exercises were not carried out in the same environment or at the same time. These factors have to be considered before making particular findings generalizable. According to Creswell (2008), experimentation in research has to be rigorous in order to make the results generalizable.
Conclusion
The findings in “The Effect of Accuracy of Instruction, Teacher Delivery and Student Attentiveness on Musicians’ evaluation of Teacher Effectiveness” call for a mixture of opinions since some attributes of the research were compared to past findings and are therefore reliable and generalizable. Nevertheless, since the research was based on human attributes that cannot be measured, there is need to do more similar researches in order to come up with more conclusive results about the phenomena under study.
References
Cohen L, Manion, L. & Morrison, K. R. B. (2003). Research Methods in Education. New York: Routledge
Creswell, J. W. (2008). Educational Research: Planning, Conducting, And Evaluating Quantitative and Qualitative Research (3rd ed.) Upper Saddle River, NJ: Merrill
Denzin, N. K. & Lincoln, Y. S. (2005).The SAGE Handbook of Qualitative Research. London: SAGE
Thyer, B. A. (2001). The handbook of social work research methods. London: SAGE,
Weeks, K. L. (1991). A comparison of seventh and eleventh-grade student perceptions of music teacher effectiveness. Unpublished master’s thesis, University of Massachusetts at Lowell
Young, S., & Shaw, D.G. (1999). Profiles of effective college and university teachers. Journal of Higher Education, 70, 670-686