In research, validity and reliability are usually seen as two complementary concepts. Nevertheless, there are clear differences between the two. For instance, validity is broadly defined as “the extent to which [the test] measures what it was supposed to measure” (John, 2015, p. 68). John (2015) explains that validity is usually based on the existence of a sound correlation between the purpose of the study and its design, including a justified choice of criteria and variables, test measures, and other factors. Reliability, on the other hand, does not evaluate the design of the study, but rather the results obtained. John (2015) explains that reliability is mostly viewed as the accuracy of measurement, which is determined by the likeliness of additional or repeated testing having the same results and outcomes.
According to the most popular definition, reliability measures the extent to which an experiment or any other process that involves measuring procedures will yield the same results if repeated trials are carried out (Ferrell, n.d., para. 2). In other words, a reliable test or experiment is the one that provides consistent results every time that it is conducted, with only minor deviations acceptable. The validity of a test is an entirely different concept that concerns not only the results of the study but the overall design and performance. Ferrell (n.d.) states that “Validity is the extent to which the construct measures what it says it is measuring” (para. 3). Therefore, validity applies to the chosen measures of the variables and whether or not they are indeed applicable to the purpose of the experiment. Reliability, on the other hand, involves making sure that the measurements of the test are consistent each time it is conducted.
In their article “Reliability and Validity in a Nutshell”, Bannigan and Watson (2009) aim to explain and explore the concepts of validity and reliability applied to research. The authors state that “Validity is concerned with the meaning and interpretation of a scale” (Bannigan & Watson, 2009, p. 3240). In general, therefore, validity measures, if the design of the study is appropriate for its purpose and whether the conclusions drawn from the results, are indeed logical. Reliability, however, is a more specific concept that can be loosely defined as the stability and internal consistency of the study. Bannigan and Watson (2009) explain that the basic purpose of reliability is to minimize the error in measurement; it is a statistical measure of how reproducible the results of the study are, i.e. whether or not re-testing will produce the same results.
John (2015) outlines that validity is a relatively abstract concept and there are no definite precautionary measures that would ensure the validity of any type of study. Still, checking the proposed study design against the relevant content domain for the studied subject can help to promote the validity of the study. For instance, researchers could search the methods used in other studies in the same area, and make sure that the design of their study fits the overall frame. Ensuring reliability, on the other hand, is usually achieved by re-testing and correlating the scores achieved in the two trials to make sure there is a sound agreement (John, 2015).
There are many ways that researchers could make sure that their study answers to the academic requirements for reliability and validity. For instance, according to the Canadian Institute for Work and Health (IWH, 2016), the process of addressing research validity includes ensuring that the research method used is valid in the current research setting. This can be done by exploring the standards set by other researchers in the field and tailoring the study design to suit the overall picture. The primary instrument for ensuring reliability, on the other hand, is the test-retest method, where repeated tests are carried out to see if the results are indeed consistent (IWH, 2016). Another way to measure reliability is by applying the study to a new group of individuals that fit the research criteria (IWH, 2016). If the study is indeed reliable, a new group of people would produce the same result as the primary group.
Bannigan and Watson (2009) also describe the different methods for ensuring the validity and reliability of research. For instance, they show how stability is one of the primary features of reliability and can be measured by a test-retest process, where “The measurement scale under development is administered on two separate occasions to the same sample and the scores are compared” (Bannigan & Watson, 2009, p. 3239). In this case, it is also necessary to determine the acceptable level of error or deviation in the results of the two tests to see if there is an overall consistency (Bannigan & Watson, 2009). In the case of research validity, however, the authors state that there is no definite procedure to ensure that the study is valid. One of the most popular ways of determining validity, therefore, is entirely subjective, as it involves “an assessment of whether a measurement scale looks reasonable, i.e. are the items included in the scale relevant?” (Bannigan & Watson, 2009, p. 3240). Other ways to ensure validity is presenting the study to an expert panel for critical review, or by comparing the study design to existing research on the same subject (Bannigan & Watson, 2009).
Annotated Bibliography
Ritter, C., Polnick, B., Fink, R. II, & Oescher, J. (2010). Classroom learning communities in educational leadership: A comparison study of three delivery options. Internet and Higher Education, 13(1-2), 96–100.
This study aimed to compare the perceptions of educational leadership graduate students regarding the effectiveness of their communication in building a classroom community. The researchers ensured the validity of the study by using the Classroom Community Scale as the major measurement tool. The CCS use was based on previous literature in the field of the classroom community, which means that the tool applied to the topic and area of inquiry. The authors also completed a principal components confirmatory factor analysis to ensure the consistency of the scale and subscale scores. Overall, the authors’ consideration of reliability and validity was adequate for the chosen topic and study design.
Sider, S., & Jean-Marie, G. (2014). Educational leadership in Haiti: A case study of innovative and exemplary leadership in a fragile state. Planning and Changing, 45(3-4), 261–284.
This study aimed to evaluate the examples of innovative and exemplary leadership practices used by school leaders in Haiti. The authors used previous literature on the subject to determine an appropriate method for their study, which increases the overall validity of their research. However, no clear attempts were made to address the reliability of the study. The results are provided in written form only with no statistical analysis or overview, which makes it difficult to address the study as a reliable source in further research.
Fine, J. B. & McNamara, K. W. (2011). Community redefined: School leaders moving from autonomy to global interdependence through short-term study abroad. Frontiers: The Interdisciplinary Journal of Study Abroad, 21, 254-274.
In this study, the authors aimed to determine the impact of short-term study abroad on the approach used by school leaders in their work. The authors address the validity of their research by clearly outlining the purposes of the study and using appropriate proven tools of measurement, such as the Global Perspective Inventory. They provide a clear rationale for the use of GPI, specifically explaining how it applies to the study. The authors also compare the results obtained in their research with results from a similar study to determine if there is consistency. Thus, the authors’ approach to promoting validity is quite thorough; nevertheless, the reliability of the study could be improved by conducting re-tests.
References
Bannigan, K., & Watson, R. (2009). Reliability and validity in a nutshell. Journal of Clinical Nursing, 18(23), 3237-3243.
Ferrell, B. (n.d.) Reliability & validity defined. Web.
Institute for Work & Health (IWH) (2016). What researchers mean by… validity and reliability. At Work, 84(1), 2.
John, A. C. (2015). Reliability and validity: A sine qua non for fair assessment of undergraduate technical and vocational education projects in Nigerian universities. Journal of Education and Practice, 6(34), 68-75.