Inter-rater reliability is explained in many different ways depending on the individuals’ field of study or operation. Despite the diverse ways in which inter-rater reliability is defined, all of the proposed definitions share certain characteristics or terms. The most important thing is that they almost altogether have the same meaning of the extent of agreement between/ among raters in the rating of either appropriateness of equipment, events and/ or performance. In simple terms, inter-rater reliability is a general class of reliability estimates, employed in the assessment of the degree to which various raters/observers provide consistent result estimates for a particular event or trend. It may involve two or more participants working independently in assigning categorical variables of a given assignment to their various categories. As a measure of reliability, it is a crucial one in the determination of the appropriateness of implementing a certain measurement system or coding (Carmines and Zeller, 2010).
Based on the knowledge and understanding gained from the theory of reliability, the measure or value of inter-rater reliability cannot be exactly calculated. However, the inter-rater reliability estimate value is evaluated/ determined by getting the scores of two or more raters to test. Two or more ways of testing the scores are used. For instance, in one event, each rater using a specified scale, maybe 1 to 20, assigns a score to each of the test items provided. Using the resulting scores of the raters, a calculation of correlation is carried out to find out the correlation between/ among ratings, hence, determining the extent to agreement (inter-rater reliability). Another way of assessing inter-rater reliability is by getting the scores of the different observations that fall in the various categories, from where percentage calculations are done to get the inter-rater reliability measure. For example, in a case where raters agree 7 out 10 times, this gives 70% of the inter-rater reliability value. There are even many more other scientific means which can be used, but there are still crude and less-scientific ways that are used and in operation. One of these crude ways is the discussion approach used with large numbers of raters with limited time. The individuals hold discussions among themselves to come up with a consensus of application, or not to apply a particular rating to an item or attribute. This is crucial in setting up guiding rules for participants in the assignment of scores, besides acting as a method of training and educating them. This is the best way in correcting the arising differences in score assignment by the individuals since helps in the elimination of such a problem in future events (Carmines and Zeller, 2010).
Focus group(s) is among the many methods used by researchers in the collection of data from different settings of our social life. The method can be used to gather data either from a population of patients, families, or even from the entire community set up. The method is used with many qualitative research topics aimed at revealing information necessary for evaluations, assessment of needs, development of specific instruments, intervention, and also in the development of concepts frameworks. Thus, this is a method that gives data qualitatively in nature. Despite the wide settings of the application, it has a general unified way in its applications, although it doesn’t mean it lacks some minor disparities (Burns and Grove, 2009).
To use focus groups in a research setting, the researcher first identifies the targeted group individuals having the characteristics and with the potential to provide the type of data wanted as it coincides with the research question and objective(s). Focus discussion groups are formed through the responsibility of the moderator and his/her assistant, who lead the group meetings in discussing the selected topic of interest within the set duration. During the discussion, recording is done manually or electronically by taping it, and then, transcribing it later to use it in the step of data analysis. Several group discussion interviews are held and recorded, with each discussion session having its own issues relevant to the research topic and aims. Finally, the recorded data is analyzed for the production of trends both for a single interview session/group and even the whole set of groups that participated.
Reference
Burns, N. and Grove, K. (2009). The Practice of Nursing Research: Appraisal, Synthesis, and Generation of Evidence. MO: Elsevier Saunders.
Carmines, E. and Zeller,.A. (2010). Reliability and validity assessment. NP: Sage Publications.