Introduction
The National Student Survey (NSS) was enforced in the year 2005 for all the Higher Educational Institutes (HEI) (Hewson 2011). NSS captures the performance and popularity of educational institutes and courses using the questionnaires’ data collected from the graduating final year students across different territories (Chalmers, Lee & Walker 2008). It comprises of 22 core questionnaires grouped under seven major categories to capture “Teaching”, “Assessment and Feedback”, “Academic Support”, “Organisation and Management”, “Learning Resources”, Personal Development”, and “Overall Satisfaction” (Surridge 2008; Douglas et al. 2015).
These questionnaires provide students’ perspective on their experience during their study. Government agencies use this data to produce a performance table listing all the participating universities, and it also provides a league table listing positons of various subject groups within a University (Gibbons, Neumayer & Perkins 2013; Chevalier & Jia 2015). The performance league table provides a useful summary to the prospective undergraduate students which in turn help them to select a particular University and course (Hahessy et al. 2014).
The University of Manchester (UoM) has been pushing frontiers of knowledge through its quality research and teaching works. In order to maintain itself among the eminent universities, UoM uses its three core goals as guiding line (The University of Manchester’s strategic plan; Georghiou 2015). The three core goals are:
- World class research,
- Outstanding learning and student experience,
- Social responsibility.
UoM stresses to providing a quality learning experience to students as stated in its second goal. The same reflects in the Key Performance Indicator (KPI 6) which states the aim of the university: “To achieve sector benchmark for Q22 in the National Student Survey by 2015 and at least 90% student satisfaction by 2020, ensuring that the University is in the upper quartile of Russell Group institutions by then.”
The student satisfaction level as measured by the NSS Q22 overall satisfaction questionnaire has increased from 3.9 in the year 2010 to 4.1 in the year 2016 (Figure 1). Though the NSS data from 2010 shows a consistent increase in the student satisfaction level of UoM, the University still falls behind in achieving 90% student satisfaction (HEFCE 2016).
Attempts have been made to generate more general measuring index; some research works have suggested the at present NSS model only measure student satisfaction level of student studying in the course; it does not include other factors such as many students drop out of course. Nevertheless, efforts are being made to make NSS a comprehensive survey. The score in NSS impact inflow of new students into a given University. Therefore, UoM needs to ensure constant improvement in student satisfaction level. This report presents a analytical view on results published from the year 2010-2016.
This report investigates the NSS data from the year 2010 to 2016, with the primary aim to propose methods to achieve the KPI6 2020. In this regard, this study interprets the NSS results using statistical parameters and tests. This report provides the answers to following three questions:
Results and Discussion
This report carried out a systematic investigation of the NSS data from the year 2010 to 2016 with a special focus on the performance of UoM with respect to HEI. In this respect, SPSS Modeler was used for statistical analysis and table generation. In addition, Excel was used for generating some of the graphs and Figures. Now, this section presents the results to address the three identified questions as listed in the previous section.
First, this report presents the percentage of students who agree on positive study experience during their study at UoM. Figure 2 shows the in the year 2010 79% students had a positive experience at UoM, which increased to 85% student with a positive experience by the year 2016. However, at present stage, it falls behind its target of 90% satisfied students by the year 2020. Further in-depth analyses of results are presented in the following sections.
- Q1.1 Has the UoM achieved the sector benchmark for Q22 in NSS in the year 2015? What about the other years?
According to NSS 2015 data, the sector-wide benchmark for Q22 was 86%, and UoM achieved exactly 86% students agreeing to positive experience. Table 1 shows that except 2015 the satisfaction level of students at UoM has been lower than the overall HIE. For example, in the year 2010, 2012, and 2013 the Q22 sector benchmarks were 83%, 85%, and 86%, while the UoM Q22 satisfaction levels were 79%, 83%, and 85% respectively.
Nevertheless, the results also show that the satisfaction level of UoM students have constantly risen from 2010 to 2015, except in the year 2016 where NSS data shows a slight dip in the satisfaction level of UoM students. In the year 2016, UoM fell behind the sector benchmark for Q22 in NSS.
Table 1: Comparison of Q22 overall satisfaction among students at UoM and overall satisfaction level of students in HIE for the year 2010-2016.
- Q1.2 In relation to the other NSS questions, what are the areas that UoM exceeds or fails to meet the sector benchmark?
Table 2 shows mean score and percentage positive agreement of students from UoM and sector-benchmark from the year 2016. Table compares two parameters to investigate the performance of UoM against the benchmark. The two parameters selected for this comparison are mean-score and percentage agreeing to a positive experience. The NSS questions are marked green where UoM performed better than the sector benchmark.
Results show that based on mean score, UoM performed better in questionnaires related to academic support (Q10-Q12), organization and management (Q13-15), learning resources (Q16-Q18), and personal development (Q19-Q21). On the other hand based on the percentage of students agreeing to a positive experience, UoM performed better than benchmark for following questionnaires i.e. organization and management (Q13-Q15) and learning resources (Q16-Q18). Therefore based on the NSS results it can be concluded that UoM failed to reach benchmark score in following areas namely:
- The teaching on my course (Q1-Q4);
- Assessment and feedback (Q5-Q9);
- Academic support (Q10-Q12);
- Personal development (Q19-Q21);
- Q22. Overall Satisfaction.
On the other hand, UoM performance exceeded sector benchmark for following NSS questions:
- Organisation and management (Q13-Q15);
- Learning resources (Q16-Q18).
Table 2: UoM performance comparison with the sector benchmark for NSS questionnaires.
- Q2. Over time, what are the programmes that have achieved the 90% target for Q22? Could you identify characteristics of the programmes that are below the target and programmes that are over?
Table 3 lists down all the programs that have achieved the 90% target for Q22. The programs showing lower than 90% satisfaction comprises of the courses offered in the field of engineering and medicine. In addition, students’ satisfaction level is higher in courses that offer a mix of specialisation. For example, students prefer BA (Hons) in Ancient History and Archaeology instead of the course such as BA (Hons) in History or BA (Hons) in English. On the other hand, the NSS Q22 results also show that new emerging courses need to stress more on improving students’ satisfaction level in comparison to traditional courses which are well established.
Table 3: List of courses having achieved 90% target for Q22.
- Q3.What actions would you propose to the university to take in order to meet KPI6 by 2020? Assuming that these actions were implemented, how you would propose to monitor progress towards KPI6 and determine whether these actions have worked?
The NSS data shows that UoM has constantly increased the student satisfaction level. For example, the Q22 questionnaire showed 79% student satisfaction in the year 2010, and it increased to 86% in the year 2016 with a marginal dip of 1% in the year 2016. However, UoM needs to take proactive measures to meet the KPI6-2020, which aims at 90% student satisfaction. A correlation analysis was conducted on the effect of preceding 21 questions to identify areas that need improvement.
As discussed earlier, NSS questionnaire checks six major aspects of the study to measure student satisfaction level. The six major areas that contribute to Q22 score are the teaching, assessment and feedback, academic support, organization and management, learning resources, and personal development. Because these six areas contribute towards overall Q22 student satisfaction, this report looked at the correlation values among these parameters. Table 4 shows correlation values of the NSS questionnaires.
Data analysis shows that UoM performance score of Q22 is strongly correlates with three parameters namely the teaching, assessment and feedback, and academic score with respective correlation values of 0.94, 0.94, and 0.95.
Now, these values show a very strong correlation; therefore, it can be concluded that NSS performance score of UoM depends on these major aspects. Further, the individual student satisfaction levels for teaching, assessment and feedback, and academic support were found to be 86%, 67% and 80% respectively (Table 2). Further, a UoM can work to improve the modules related to personality development of individuals because the NSS data shows lower student satisfaction.
It should be noted that both academic support and personality development are strongly correlated. Therefore, improving the academic staff and structure would help in higher student satisfaction with regard to both personality development and academic support. Assuming UoM implement the changes suggested above to improve students experience, it is vital to keep monitoring the progress of proposed parameters. In this respect, UoM need to ensure keeping a track on the teaching quality.
The correlation analysis and discussion in this section show that strong correlation exists between the Q22 score and the three identified core areas. In other words, UoM needs to focus on improving the student teaching experience, assessment and feedback experience, and provide better academic support to the students. NSS data analysis reveals that UoM is performing above the sector-side benchmark in learning resources, and organisation and management. However, these areas are less correlated to Q22 score. Hence, UoM needs to focus on the teaching, assessment and feedback, and academic support.
Table 4. Correlation coefficient for Q1-Q22 NSS questionnaires.
Conclusion
Analysis of NSS data from the year 2010 to 2016 shows a dynamic improvement in student satisfaction at UoM. This reflects in the data where only 79% students were satisfied in the year 2010, and the student satisfaction improved to 86% in the year 2015 with a 1% dip in the year 2016. Though the consistent increase in student satisfaction shows UoM efforts in the preceding years, there is much work required to meet its KPI6-2020 goal to achieve 90% student satisfaction.
Results from the analysis also show that UoM is performing on par and in some cases above the sector-level benchmark. For example, UoM scores above the sector-wide score in organisation and management, and learning development. However, in order to meet the KPI6-2020 goal, UoM needs to give special attention to students experience in assessment and feedback. NSS data shows only 67% students were satisfied in the year 2016 with the present assessment and feedback process. In addition, the University should focus on the teaching and academic support. At present 80% students satisfaction was recorded for the teaching and academic support.
Nevertheless, the Q22 satisfaction score shows a strong correlation with the teaching and academic support. Hence, there is need to focus on improving three major areas namely the teaching, assessment and feedback and academic support. The NSS feedback adds greater depth to working goals of Universities across region. A more robust analysis should be conducted to identify other weak areas which can be improved to help UoM reach the overall objective of KPI6-2020. In addition, as suggested regular monitoring must be carried out to ensure the programs are on track and student feedback improves sufficiently each year. For example, as suggested in the previous section, there is need to work on inter-related factors such as personality development and academic support.
Reference List
Chalmers, D, Lee, K & Walker, B 2008. International and national quality teaching and learning performance models currently in use. Carrick Institute for Learning and Teaching in Higher Education, Western Australia.
Chevalier, A & Jia, X 2015, ‘Subject Specific League Tables and Students’ Application Decisions’, Manchester School, vol. 84, no. 5, pp. 1-21. Web.
Douglas, JA, Douglas, A, Mcclelland, RJ & Davies, J 2015, ‘Understanding student satisfaction and dissatisfaction: an interpretive study in the UK higher education context’, Studies in Higher Education, 40, pp. 329-349.
Georghiou, L 2015, ‘Strategy to Join the Elite: merger and the 2015 Agenda at the University of Manchester – an update’, in A Curaj, L Georghiou, AC Harper, E Egron-Polak (eds), Mergers and Alliances in Higher Education, Springer, pp. 205-220.
Gibbons, S, Neumayer, E & Perkins, R 2013, ‘Student satisfaction, league tables and university applications‘, Discussion Paper, London School of Economics, London. Web.
Hahessy, S, Burke, E, Byrne, E, Farrelly, F, Kelly, M, Mooney, B & Meskell, P 2014, ‘Indicators of student satisfaction in postgraduate blended learning programmes: Key messages from a survey study‘, The All Ireland Journal of Teaching & Learning in Higher Education, vol. 3, no. 4. Web.
HEFCE 2016, National Student Survey 2010-2016. Web.
Hewson, P 2011, ‘Preliminary Analysis of the National Student Survey’, MSOR Connections, vol. 11, no. 1, pp. 25-28.
The University of Manchester’s strategic plan. Web.
Surridge, P 2008, The National Student Survey 2005-2007: findings and trends, HEFCE, London.