For a long time, many educational institutions have been generating data about student performance and storing them manually. Without a proper plan on how to use such information for the advancement of educational outcomes, some teachers have failed to acquire the requisite knowledge needed to explain how such data can be used to improve decision-making skills in the learning setting (Warschauer, 2020). This problem stems from the reliance on traditional/manual data management methods, which failed to provide a framework for teachers to integrate available data in their decision-making processes. Additionally, the lack of fluidity in the management of manually stored data has made it difficult to track a student’s educational progress. This problem cuts across different types of educational services because tracking and monitoring a student’s progress is an imperative process in learning (Prinsloo, 2017). To address it, data-driven evaluation techniques have been introduced to help education stakeholders make better and informed decisions regarding their teaching practice.
In this paper, the role of data-informed decision-making in improving the learning outcomes of students who study English as a Foreign Language (EFL) is explored. The discussion is presented in two parts. In the first one, the theoretical background of the study is explored with three issues discussed: understanding the role of Information Technology (IT) in EFL, evaluating the importance of acquiring data-driven decision-making skills in education, and evaluating the extent that information systems have been used to improve the outcomes of EFL teaching and learning. In the second stage of analysis, a context of the overall discussion will be provided using the English Language Institute as a case study for the implementation of IT tools in decision-making. A comprehensive description of the system adopted in EFL will be provided in the same section and discussions/thoughts on its reliability outlined in a new section of the study. In the last section of the study, the information highlighted above will be collated, analyzed, and used to justify recommendations to improve the institution’s evaluation information system.
Theoretical Background
IT in Teaching and Learning English as a Foreign Language
The use of data in decision-making is not a new phenomenon in the education sector because instructors have always used traditional tools of assessment, such as conducting physical tests and using the findings to develop instruction practices. These traditional tools of assessment used data derived from a teacher’s educational experiences, intuition, and teaching philosophy to make critical decisions about a learner’s educational progress and development (Savitz-Romer et al., 2018). The use of IT-enabled tools for data assessment is regarded as a new approach to making decisions in the education setting because it emphasizes the need to use data-driven information to formulate and implement educational policies. These tools of assessment encourage educators to use empirical knowledge to make decisions about the teaching practice (Mertler, 2020). They are a superior way of managing information because they could lead to better decision-making based on their reliance on quality and actionable data.
Over the past few decades, there has been an increase in the pace of technological adoption in the education sector. Many activities and processes in the field have benefitted from this development and the study of English as a foreign language is no exception (Sampson, 2019; Fenwick & Edwards, 2016). However, differences between traditional and IT-enabled data analysis tools have created differences in their impact on EFL learning. For example, old data assessment tools have had a lower effect on EFL learning compared to the use of IT-enabled assessment procedures because they have failed to create systematic processes for monitoring and evaluating data. Comparatively, IT-enabled tools have allowed educators to enjoy these advantages and much more by providing a standardized way of collecting, analyzing, and evaluating information (Piety, 2019). In this regard, IT is a revolutionary tool for data management and evaluation within the EFL space.
Based on the superior role played by IT in facilitating data evaluation processes in EFL learning, it has become increasingly clear to educators that traditional methods of information assessment are vulnerable to human errors, which may affect students’ learning outcomes (González-Sancho & Vincent-Lancrin, 2016). This is because the unstandardized nature of manual evaluation systems makes it difficult to determine which strategy to use in an education setting and the measurement criteria to use in monitoring progress. In other words, traditional data assessment tools do not provide a framework for evaluating information holistically because experiences vary across groups of teachers and institutional settings. IT-enabled tools have helped to address this problem by providing a platform for educational institutions to use their resources to develop a standardized software or system that appeals to their specific needs and dynamics (Nieminen & Hyytinen, 2015). Therefore, IT has created flexibility in the evaluation of educational data. However, the integration of IT tools in teaching English as a foreign language has been adopted using the e-learning framework.
The e-learning model involves the use of computer-enabled teaching methods to facilitate EFL learning. In most research studies, this concept has been associated with the use of the internet as a mode of teaching English as a foreign language (Mutambik, 2018; Yıldırım, & İspinar, 2019). Stated differently the internet is regarded as a principal or supplementary education resource for EFL learning. The case for the use of IT in learning English as a foreign language has been made by highlighting the power of technology in eliminating traditional barriers to education, such as geographical and time differences. Researchers have also pointed out that the use of IT tools in learning English as a foreign language is also rooted in its ability to eliminate spatial and temporal challenges to learning (Ellison & Aloe, 2019; Zhang et al., 2016). Other researchers have pointed out that the use of IT in EFL learning has helped to provide students with an increased array of language resources needed to communicate more effectively with their teacher and colleagues (Hartong, 2016; Williamson, 2016a; Williamson, 2016b; Souto-Otero & Beneito-Montagut, 2016). These competencies have been captured by technology integration theories used in the education sector, such as the intentional use of technology model, which explains how IT tools can be used to assess and monitor students’ learning outcomes. It suggests that three categories of technology are used in helping teachers to make sound judgments about students’ learning outcomes: service, engagement, and learning (Steele, 2015). IT enhances these areas of evaluation in EFL learning and highlight the importance of data-driven decision making in education.
Importance of Data-Driven Decision Making In Education
Empirically informed decisions emanate from the use of IT-enabled tools in information gathering and assessment. Consequently, data-driven decision-making has been at the center of educational reforms since the adoption of the No Child Left behind Program conceptualized in the US (Jung & Young, 2019). Since the inception of the program in 2001, it has become routine for children across all grade levels to complete standardized tests and the same practice has been borrowed in EFL teaching (Dunn, 2016). As teachers measure the yearly progress of all children involved in associated educational programs, educators must embrace data-driven decision-making processes to understand what students know and what they are yet to master (Williamson, 2017). Doing so will help them to understand the learning gaps that exist and how to fill them.
As its name suggests, data-driven decision-making is focused on using empirical information to make important decisions affecting a student’s educational progress. The data used is instrumental in choosing what to teach and when to teach it. Therefore, instead of looking at teaching as a form of art that requires a teacher’s intuition to make instructional choices in learning, it is presented as a process that uses empirical data to minimize the negative effects of trial and error approaches in educational decision making.
The formalization of education has seen a long-term trend where teachers and educators have strived to use data to inform their decisions. Those who have succeeded in this regard use information from different sources to come up with their educational plans. However, recently, the development of IT tools and their integration in EFL learning have broadened the scope of information available for review through the rapid generation of data (Conaway et al., 2015). Indeed, these tools allow educators to generate data in real-time and assess them in the same fashion. In this regard, it is beneficial to teachers and educators alike because it simplifies their data assessment needs and requirements.
Data-driven decision-making is an important process in the improvement of student learning outcomes. For example, it has been used to evaluate educational progress among students and keep track of changes that have occurred in the course of a student’s educational journey (Prinsloo, 2020). Additionally, teachers have used it to better plan their educational curricula and identify areas requiring improvement by monitoring and integrating information relating to a student’s educational progress. To have the maximum possible effect on their educational outcomes, the use of data-driven decision-making tools in EFL learning has spanned several cadres of educational assessment (Prinsloo, 2020). They also involve different players, including students, non-teaching staff, and teachers.
Overall, the use of data-driven techniques in EFL learning provides educational stakeholders with a basis they can use to track changes in a student’s educational journey and provide a longitudinal record of information needed to understand their educational experiences and how to improve them in the future. Overall, data-driven decision-making processes are beneficial in EFL learning because it helps them to make informed decisions about the teaching practice, based on empirical evidence. These decisions can be used to improve EFL learning outcomes using information system assessment tools.
Improving Outcomes of EFL Teaching and Learning through Information Systems
Data can be used to improve teaching and learning outcomes if implemented correctly in EFL learning. The process of data assessment and integration is often characterized by a 5-step cycle that involves activities centered on reviewing past practices, devising a plan of action to address important areas of attention involving the past practices, implementing the plan of action, assessing and measuring outcomes, and transforming data into actionable information. These strategies of data assessment and analysis are often closely linked with one another and may overlap at different stages of analysis.
The aforementioned cycle is commonly used to explain processes surrounding data treatment and evaluation geared towards improving outcomes of EFL teaching and learning through the adoption of information systems assessment techniques. Relative to this assertion, EFL teaching processes have been affected by concerns regarding the methodologies chosen to assess learning outcomes (Huang et al., 2019). For example, some researchers have expressed concern regarding the overreliance on academic rewards mechanism to improve educational outcomes because there is enough evidence to suggest that using academic success as the main metric of evaluation does not necessarily improve educational outcomes (Nagy, 2016; Fischer, et al., 2020). To address this problem, educational stakeholders need to make a concerted effort to provide a holistic framework for evaluating educational outcomes. It should not be biased against students who do not have “academic buoyancy;” instead, it should reward those who make incremental achievements.
The holistic framework of evaluation should create a space for nurturing students to develop different competencies in language development by outlining a set of key performance indicators to be used to evaluate educational outcomes. However, to realize the above-mentioned outcomes, there should be an individualized culture of evaluating student outcomes in EFL learning over a long period of assessment. Studies have shown that parents hold a more favorable view of this type of review because it helps them to understand the unique learning needs and requirements of each student, as opposed to one that measures their performance using a generalized framework or standard of assessment (Hobsons, 2014). This statement is further supported by the fact that most parents prefer to hear the educational progress of their children before understanding how it compares with those of others in the education setting.
Overall, a data-driven education system will strive to evaluate a students’ educational progress from the time they enter the education system to the time they leave. It offers a comprehensive understanding of students’ educational outcomes through a succinct monitoring system that is predicated on collecting data and analyzing them based on a set of key performance indicators. Performance comparisons are often undertaken with a careful understanding of the evaluation context. The strengths of existing IT systems in appealing to the contextual needs of each EFL learner or institutional setting is also captured by understanding the context of evaluation.
Context
The English Language Institute
Over time, many institutions offering English learning as a foreign language have done so to help institutions meet their diversity enrolment requirements. These requirements stem from the need for some universities to maintain a multicultural image (Fernández-Agüero & Chancay-Cedeño, 2019). Therefore, they are likely to enroll students who do not speak English as their first language and would want to learn it to further their educational career. The English Language Institute provides an academic setting where students can easily learn the language and socialize with colleagues who have the same goal. This educational context is situated within the larger institutional setting where other educational activities, besides English learning, occur. Most students who are enrolled in the English Language Institute are people who are studying the language as a minimum requirement for the pursuit of their educational programs. Others participate in the program to expand their breadth of linguistic skills and abilities.
The English Language Institute provides students with a unique learning environment where they are subjected to an intensive learning curriculum and linguistic exercises that allow them to practice speaking the language in a real-world setting. Dang et al. (2020) say that this type of environment has a positive effect on student learning outcomes. Its appeal in EFL learning is rooted in the fact that the English Language Institute is committed to respecting diversity and equity as recommended by Toh (2015). The English Language Institute’s education evaluation system was developed after assessing the test scores of several students. The decision to use an IT evaluation system is rooted in the institution’s capability to illuminate the decisions of its major stakeholders. A comprehensive description of the proposed system is outlined below.
Comprehensive Description of the Proposed System
The education evaluation system at the English Language Institute has five main users: management staff, system administrators, coordinators, teachers, and students. The model has different subsystems that can be accessed by students, staff, and authorities. The management staff is responsible for all processes, systems, and procedures associated with the model, while the admin is the most powerful entity within the institution. The admin is a “superuser,” who wields veto power over all decisions made within the broader system setup. This power is exercised in the running of the education evaluation system because the admin can access the profile of any user and view their details. They can also use gain access to information relating to timetables, classes, and rosters. Using this administrative privilege, they can also delegate roles for team members, including determining who will be the chief coordinator and evaluator before the production or preparation of the final report.
The system admin works under the authority of the management staff because they control all information systems. Only one admin has immense power to control the system, thereby creating a centralized point of command that runs all technical activities in the system. In this regard, they have control over key activities that underscore the system. For example, they could create, edit, or delete any type of data from the system. They could also access the profile of any user who uses the system and amend their information, if necessary. They can also monitor all details relating to students’ performances, including determining whether they are related to staff outcomes, or not. The admin also has unfettered access to data used in assigning students to their respective classes and information needed to assign teachers who will be monitoring their educational progress. The administrator also has to upload all timetables and class rosters guiding student and teacher activities.
The coordinator is also an important entity in the overall evaluation system because he supervises and monitors all management processes. Some key tasks undertaken by them include viewing student and teacher profiles, reviewing marking components, attendance records, timetables, coverage plans, grading schemes, and evaluation reports. Teachers are also another important group of users in the evaluation system, who enjoy special privileges to perform important tasks, such as marking attendance, assigning and checking homework, and viewing students’ profile. Lastly, students also play an important role as end-users in the proposed system, with their main task being viewing and uploading tasks on various discussion platforms, accessing timetables, viewing grades, marking students’ attendance records, and communicating to the respondents regarding grievances that may impact the overall learning process. From the student’s perspective, basic functions they can undertake in the system include login, changing passwords, updating personal information, and logging out.
The overall structure of the evaluation information system is mainly Internet-based and acts as a central repository of all data. Multiple groups of people, including students, authorities, and educational staff, can access the system. These parties can access existing data through highly refined interfaces designed to suit their specific needs. The model offers a tracing system where teachers can evaluate students’ progress. These features provide consistent measurement statistics, which can be used to quantify the progress made by each student or a cross-section of them. Some key features offered by the system include student registration, records maintenance, timetable generation, and the development of progress reports.
An automated evaluation review framework means that authorities can better monitor students’ performance and behavior. This will happen through the generation of reliable data relating to a student’s academic or educational performance. For example, information about a student’s attendance record, schedules, grades, and achievements are generated in this manner and such data used to identify areas requiring improvements. Teachers benefit the most from the utilization of such information because they can monitor student performance in real-time. Indeed, by clicking on a few items on the keyboard, they could gain access to real-time data and make follow-ups that are essential in monitoring performance. For example, they could use the same platform to compare historical averages of educational performance among groups of students.
Management can also use the information generated from the evaluation processes to evaluate the performance of both students and teachers. In this regard, they have more data needed to make decisions or policies that affect both parties. For example, they could use such data to improve resource allocation decisions affecting both students and teachers. Alternatively, they could use such information to benchmark student and teacher performances based on a specific set of key performance indicators. Some key features of the proposed system include a multi-user account system, a friendly interface, grading, daily attendance, internal messaging, announcements, calendar, assignments, complaints, evaluation, dashboard, log viewer, and CBT scheduling. Key objectives of the proposed system include assessing learners’ performance and progress continuously, improving documentation and security, seamlessly managing student/teacher data from admission until exit, enabling teachers to support students’ success, analyzing performance based on any measurable key performance indicator, and promote a local intervention environment.
Discussion and Thoughts on the Proposed System
So far, data generated from the above-mentioned system provides a reliable basis for making changes to its overall design because it has been operational since 2018. However, the presence of too many users in the system may compromise its integrity if a security breach occurs. For example, the system is accessible to management staff, system administrators, coordinators, teachers, and students. The danger in having many of these users is rooted in the fact that that there are too many entry points of information access. If they are maintained and allowed to operate as they are, the potential for a data breach is high, from a security analysis perspective.
Alternatively, although groups of stakeholders operate the evaluation system in the institute’s education system, teachers’ responses regarding different aspects of student’s learning are mainly predicated on their ability to extract data from complex charts and graphs. Therefore, those who do not have advanced data analytical skills may have a problem performing such tasks. This limited ability to evaluate data means that there are constraints on the ability of educators to use data in their decision-making processes. Overall, some of the recommendations that will be outlined in the next section of the paper will highlight solutions that will address these issues.
Recommendations
The presence of too many entry points in the school’s education evaluation system has been highlighted in this study as one of the key concerns affecting its integrity. Notably, there are five points of information access, which are operationalized by management staff, system administrators, coordinators, teachers, and students. This five-point model can be collapsed into a three-tier data access system whereby management staff, system administrators, and coordinators are given one access point because their roles overlap with one another and are related. In other words, they mostly perform supervisory and administrative tasks, which could be merged, into one pool of information access. Data access privileges accorded to teachers and students can remain as they are because they are two distinct parties in the overall educational system. Therefore, the overall system should have three data access points, which should be operationalized by management, teachers, and students. Doing so would improve the integrity of data obtained and stored in the system, thereby improving the integrity of findings generated. These changes should also improve the design of the overall system from a security perspective.
There is also a need to engage teachers in further training to improve their data management skills. Doing so will improve their ability to better use IT tools that support the evaluation process and interrogate new information obtained from the evaluation processes. There should also be a designated time when teachers should analyze data in small groups to avoid inconsistencies that may occur when data is collated and analyzed during different periods. In this regard, there is a strong need for leadership in the educational setting, which should provide the context of adopting such changes and help in evaluating the educational performance and record of students in the organization. Besides providing timelines for evaluation, leadership is also needed to communicate student and teacher needs.
The scale used to undertake student evaluation also needs to be changed from the 5-point to the 7-point Likert scale, which offers a broader array of options to capture respondents’ views. The 7-point Likert scale is superior to the currently used metric because it offers a greater array of choices for students to express their views. Relative to this assertion, Ferfolja et al. (2018) say that when students are provided with a slimmer array of options to express their views regarding an educational assessment issue, they are forced to choose the “next best alternative.” This situation creates measurement errors, which could cause instructors to develop erroneous judgments about their performance.
The current assessment system measures students’ responses based on five key categories of evaluation, including strongly “strongly agree,” “agree,” “neutral,” “disagree,” and “strongly disagree.” The 7-point Likert scale should introduce more options for the respondents to discuss, by adding two new options, including “somewhat agree’ and “somewhat disagree.” The 7-point Likert scale is more appropriate to use in the student evaluation process because it best captures different levels of understanding that respondents may have regarding a language issue (Ma et al., 2017). In other words, language understanding is a highly subjective issue that is best captured by giving students a wider range of options to assess their understanding of various learning concepts (Goris et al., 2019). Therefore, a richer pool of data can be obtained by adopting the measurement technique.
Overall, although this paper identifies IT-enabled tools as a powerful system of generating and analyzing educational data, it should not be deemed a substitute for old techniques of data assessment and analysis. Concisely, it should be used alongside traditional tools of data assessment because there needs to be a balance in the use of computer-generated data and teacher intuition. Particularly, the latter concept is instrumental in the dissemination and use of data because it provides a contextual understanding of information usage in EFL learning. Therefore, there needs to be a careful balance between human and IT-enabled tools when developing a robust evaluation system. These changes should be adopted because old tools of data analysis are inadequate to cover today’s pressing decision-making challenges. In other words, they ought to be coupled with newer techniques that can better handle new data nuances for better information analysis. These new tools of information analysis may involve the process of generating data from standardized tests and IT-enabled information to encompass the new digital information age characterizing EFL learning today.
References
Conaway, C., Keesler, V., & Schwartz, N. (2015). What research do state education agencies really need? The promise and limitations of state longitudinal data systems. Educational Evaluation and Policy Analysis, 37(1), 16–28.
Dang, T. N. Y., Webb, S., & Coxhead, A. (2020). Evaluating lists of high-frequency words: Teachers’ and learners’ perspectives. Language Teaching Research, 4(1), pp. 1–10.
Dunn, K. E. (2016). Educational psychology’s instructional challenge: Pre-service teacher concerns regarding classroom-level data-driven decision-making. Psychology Learning and Teaching, 15(1), 31–43.
Ellison, S., & Aloe, A. M. (2019). Doing identity work and risky endeavors? A qualitative research synthesis of predominantly white, middle-class parents’ decision making in the context of urban school choice. Education and Urban Society, 51(1), 72–98.
Fenwick, T., & Edwards, R. (2016). Exploring the impact of digital technologies on professional responsibilities and education. European Educational Research Journal, 15(1), 117–131.
Ferfolja, T., Diaz, C. J., & Ullman, J. (2018). Understanding sociological theory for educational practices. Cambridge University Press.
Fernández-Agüero, M., & Chancay-Cedeño, C. (2019). Interculturality in the language class – Teachers’ intercultural practices in Ecuador. Regional Language Centre Journal, 50(1), 164–178.
Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., Slater, S., Baker, R., & González-Sancho, C., & Vincent-Lancrin, S. (2016). Transforming education by using a new generation of information systems. Policy Futures in Education, 14(6), 741–758.
Goris, J., Denessen, E., & Verhoeven, L. (2019). Effects of content and language integrated learning in Europe a systematic review of longitudinal experimental studies. European Educational Research Journal, 18(6), 675–698.
Hartong, S. (2016). Between assessments, digital technologies, and big data: The growing influence of ‘hidden’ data mediators in education. European Educational Research Journal, 15(5), 523–536.
Hobsons. (2014). Student feedback and progressive reporting. Hobsons Australia Pty Ltd.
Huang, F., Teo, T., & Zhou, M. (2019). Factors affecting Chinese English as a foreign language teachers’ technology acceptance: a qualitative study. Journal of Educational Computing Research, 57(1), 83–105.
Jung, J. Y., & Young, M. (2019). The occupational/career decision-making processes of intellectually gifted adolescents from economically disadvantaged backgrounds: A mixed methods perspective. Gifted Child Quarterly, 63(1), 36–57.
Ma, W., Chan, C., Tong, K., Fung, H., & Fong, C. (2017). New ecology for education — communication X learning: Selected papers from the HKAECT-AECT 2017 summer international research symposium. Springer.
Mertler, C. A. (2020). Introduction to data-driven educational decision making. Web.
Mutambik, I. (2018). The role of e-learning in studying English as a foreign language in Saudi Arabia: students and teachers’ perspectives. English Language Teaching; 11(5), 74–83.
Nagy, R. (2016). Tracking & visualizing student effort: Evolution of a practical analytics tool for staff and student engagement. Journal of Learning Analytics, 3(2), 165–193.
Nieminen, M., & Hyytinen, K. (2015). Future-oriented impact assessment: Supporting strategic decision-making in complex socio-technical environments. Evaluation, 21(4), 448–461.
Piety, P. J. (2019). Components, infrastructures, and capacity: the quest for the impact of actionable data use on P–20 educator practice. Review of Research in Education, 43(1), 394–421.
Prinsloo, P. (2017). Fleeing from Frankenstein’s monster and meeting Kafka on the way: Algorithmic decision-making in higher education. E-Learning and Digital Media, 14(3), 138–163.
Prinsloo, P. (2020). Of ‘black boxes’ and algorithmic decision-making in (higher) education – A commentary. Big Data and Society, 9(2), pp. 1–10.
Sampson, R. J. (2019). Openness to messages about English as a foreign language: Working with learners to uncover purpose to study. Language Teaching Research, 23(1), 126–142.
Savitz-Romer, M. S., Nicola, T. P., Jensen, A., Hill, N. E., Liang, B., & Perella, J. (2018). Data-driven school counseling: The role of the research-practice partnership. Professional School Counseling, 8(3), pp. 1–10.
Souto-Otero, M., & Beneito-Montagut, R. (2016). From governing through data to governmentality through data: Artefacts, strategies, and the digital turn. European Educational Research Journal, 15(1), 14–33.
Steele, G. (2015). Using technology for evaluation and assessment. Web.
Toh, G. (2015). ‘A tale of two programs’: Interrogating ‘open (closed) ness’ and ‘cultural diversity’ through critical observations of two Japanese university English language programs. Policy Futures in Education, 13(7), 900–916.
Warschauer, M. (2020). Mining big data in education: Affordances and challenges. Review of Research in Education, 44(1), 130–160.
Williamson, B. (2016a). Digital methodologies of education governance: Pearson PLC and the remediation of methods. European Educational Research Journal, 15(1), 34–53.
Williamson, B. (2016b). Digital education governance: An introduction. European Educational Research Journal, 15(1), 3–13.
Williamson, B. (2017). Who owns educational theory? Big data, algorithms, and the expert power of education data science. E-Learning and Digital Media, 14(3), 105–122.
Yıldırım, R., & İspinar, D. (2019). A study of young gifted learners’ and their teachers’ perceptions of effective EFL learners. Journal for the Education of the Gifted, 42(1), 85–104.
Zhang, X., Anderson, R. C., Morris, J., Miller, B., Nguyen-Jahiel, K. T., Lin, T. J., Zhang, J., Jadallah, M., Scott, T., Sun, J., Latawiec, B., Ma, S., Grabow, K., & Hsu, J. Y. L. (2016).
Improving children’s competence as decision-makers: contrasting effects of collaborative interaction and direct instruction. American Educational Research Journal, 53(1), 194–223.