Home > Free Essays > Education > Aspects of Education > Fatigue Impact on Human Raters
Rate

Fatigue Impact on Human Raters Essay (Book Review)

Exclusively available on IvyPanda Available only on IvyPanda
Updated: Jan 24th, 2021

Introduction

The article “A Study of the Impact of Fatigue on Human Raters when Scoring Speaking Responses” was conducted in 2014 by Guangming Ling, Pamela Mollaun, and Xiaoming Xi. Informed by a research gap on the impact of time-related fatigue on rating quality in language testing, these authors set out to examine the effects of fatigue on human raters’ scoring performance within the context of the TOEFL iBT Speaking Test. The authors succeeded in showing how rating accuracy, quality, and consistency vary across time due to fatigue-related effects.

Summary

The study employed a quantitative experimental research design to “examine the effects of fatigue on human raters of audio responses by comparing rating accuracy and consistency at various time points throughout a scoring day, under shift conditions that differ by shift length and session length” (Ling, Mollaun, and Xi 479). The authors consulted other sources to show that the burden placed on raters’ concentration and cognitive capabilities in scoring speaking responses may trigger time-related fatigue and also aggravate scoring accuracy and consistency. Overall, the findings proved that (1) scoring productivity and quality differ widely across hours irrespective of shift conditions, (2) shorter scoring shifts (e.g., 6-hours shifts and those that switch to a new speaking task every 2 hours) enjoy higher rating accuracy, greater hourly productivity, and greater rating consistency across time than longer scoring shifts, and (3) longer scoring sessions and expanded scoring length are positively associated with increased fatigue.

Critique

The main argument of the study is that fatigue occasioned by longer scoring shift length and session length affects the human raters’ rating quality, accuracy, as well as consistency. The writers’ perspective is embedded in the fact that scoring shift length and session length in language testing may indeed occasion undesirable fatigue-related outcomes. This perspective is consistent with that of other authors, who argue that exam proximity (time between cognitive tasks) has a powerful impact on performance due to cognitive fatigue (Fillmore and Pope 12). Drawing from this comparison, it can be argued that the writers’ ideas on fatigue are similar to those of other writers. Indeed, their ideas are effective in demonstrating how scoring shift length and session length affect the rating quality, accuracy, and consistency in language testing.

The authors do not show any bias and are qualified to write in this area based on their affiliation with the United States Educational Testing Service. Their main ideas are easily agreeable based on the experiments done in the study to prove them; however, the authors fail to discuss how other factors (e.g., regional traditions of language testing) can impact rating quality, accuracy, and consistency (McNamara and Knoch 556). Although their introduction and literature review sections are easy to understand, other areas (e.g., methodology and results in sections) seem complicated to a general reader. Additionally, although the writers use reputable sources to back their arguments, they fail to connect the available literature to their main findings in a manner that could reinforce their main contributions in the area. Overall, the study can be recommended to students and professionals who may want to understand the effects of fatigue in language testing and scoring.

Conclusion

The article is effective in demonstrating how fatigue-related factors affect the rating accuracy of human raters in language testing. Language testing and scoring professionals may find this article interesting as it provides a glimpse of ideal scoring shift length and session length for optimal productivity in rating quality, accuracy, and consistency.

Works Cited

Fillmore, Ian and Devin G. Pope. The Impact of Time between Cognitive Tasks on Performance: Evidence from Advanced Placement Exams, 2012. Web.

Ling, Guangming, Pamela Mollaun and Xiaoming Xi. “A Study of the Impact of Fatigue on Human Raters when Scoring Speaking Responses.” Language Testing. 31.4 (2014): 479-499. ERIC. Web.

McNamara, Tim and Ute Knoch. “The Rasch Wars: The Emergence of Rasch Measurement in Language Testing.” Language Testing. 29.4 (2012): 555-576. ERIC. Web.

This book review on Fatigue Impact on Human Raters was written and submitted by your fellow student. You are free to use it for research and reference purposes in order to write your own paper; however, you must cite it accordingly.
Removal Request
If you are the copyright owner of this paper and no longer wish to have your work published on IvyPanda.
Request the removal

Need a custom Book Review sample written from scratch by
professional specifically for you?

Writer online avatar
Writer online avatar
Writer online avatar
Writer online avatar
Writer online avatar
Writer online avatar
Writer online avatar
Writer online avatar
Writer online avatar
Writer online avatar
Writer online avatar
Writer online avatar

certified writers online

Cite This paper
Select a referencing style:

Reference

IvyPanda. (2021, January 24). Fatigue Impact on Human Raters. Retrieved from https://ivypanda.com/essays/fatigue-impact-on-human-raters/

Work Cited

"Fatigue Impact on Human Raters." IvyPanda, 24 Jan. 2021, ivypanda.com/essays/fatigue-impact-on-human-raters/.

1. IvyPanda. "Fatigue Impact on Human Raters." January 24, 2021. https://ivypanda.com/essays/fatigue-impact-on-human-raters/.


Bibliography


IvyPanda. "Fatigue Impact on Human Raters." January 24, 2021. https://ivypanda.com/essays/fatigue-impact-on-human-raters/.

References

IvyPanda. 2021. "Fatigue Impact on Human Raters." January 24, 2021. https://ivypanda.com/essays/fatigue-impact-on-human-raters/.

References

IvyPanda. (2021) 'Fatigue Impact on Human Raters'. 24 January.

More related papers
Psst... Stuck with your
assignment? 😱
Hellen
Online
Psst... Stuck with your assignment? 😱
Do you need an essay to be done?
What type of assignment 📝 do you need?
How many pages (words) do you need? Let's see if we can help you!