Introduction
University of Delaware (UD) is a private-public research university located in Newark, Delaware, founded in 1743, making it one of the oldest universities in the nation and the largest university in the state of Delaware. UD is a large institution with its main campus in Newark hosting 17,710 undergraduate students enrolled in 2020 (IRE, 2021a.) and by definition (SAGE, 2021) considered a predominantly white institution (PWI), hosting 68.4% White student population. The following data retrieved from UD’s Institutional Research and Effectiveness (IRE) office gives context to the demographics of the population at the main campus in the year 2020 distinguishing the total percentage of students by race and ethnicity. Black or African American students are 5.7% of the population; Hispanic or Latino students are 8.8%; Asian students are 5.3%; Native American students make up 0.1%; International students are 5.3%; and Other students are 6.4% of the population, see Appendix A.
The key stakeholders for the evaluation are the Hispanic and Black students and the leadership of the UD. Specifically, the UD is represented by the Interim Vice Provost, the Center of Black Culture, the Division of Student Life, and the Equity and Inclusion team (UD, 2021a). The interest of the Vice Provost for Diversity and the UD is the improvement of retention and graduation rates through the development of an inclusive academic environment and supportive communities for the students of color.
The UD adopts a strategy that acknowledges the benefits of diversity for academic performance and fosters an inclusive approach by strengthening partnerships with other institutions, providing plentiful programs and opportunities for students, and creating a supportive learning environment. Concurrently, there are challenges associated with providing support for minority students, such as lack of resourcing, difficulties ensuring a holistic approach to the issue, and staff attitudes and practices regarding teaching minority students. Thus, the issue of retaining underrepresented students becomes specifically acute and must be addressed. In this regard, the administration’s approaches to student diversity and ethnic disparities are critical to eliminating barriers in higher education completion. Therefore, examining the current practices supporting these groups of students is essential to understanding, developing, and implementing effective strategies at UD, and increasing graduation and retention rates.
Purpose and Nature of the Evaluation
The UD’s lowest retainer in graduation for the first-time, full-time, first-year students are the Black and Hispanic population. I will refer to this student population as people of color (POC) for the purpose of this evaluation. Institutional Research and Effectiveness (IRE, 2021b.) data show the 2016 cohort of White female students graduated 83% of the class in four years, compared to 62% of Black and 70% Hispanic female peers. Additionally, 66% of White males graduated in four years, compared to Black males at 41% and Hispanic males at 58% completion rate by the fourth year, see Appendix B. There is a large evident gap in graduation rates between the White students and the POC students.
Furthermore, IRE (IRE, 2021c.) data also show that from 2014 to 2019 retention rates of POC students remained consistently lower than their White peers, see Appendix C. IRE data show the 2016 cohort of White female students retained 88% of the class to the fourth fall, compared to 80% of Black and 75% Hispanic female peers. Additionally, 82% of White males were retained through fourth fall, compared to Black males at 71% and Hispanic males at 76%. It is unclear if there is further attrition after the fourth fall, though graduation rates support further loss in retention due through the sixth year of college, see Appendix B.
Colleges and universities are challenged by improving graduation rates and are aggressively making retention and degree completion of students of color a priority (Green and Wright, 2017). The positive outcomes of diversity in education are often highlighted by research, which emphasizes the need for creating ethnically and culturally diverse educational environments (Grapin & Pereiras, 2019). Hence, examining the supportive practices to foster minority student groups’ retention and graduation will provide valuable insight and identify aspects for further improvement. Supportive practices are vital for providing minority students with an opportunity to overcome academic struggles and eliminate challenges throughout the educational process. Thus, this evaluation will focus on the following goals:
- Improve the current systems of connection with the minority students to increase engagement, support, and sense of community.
- Increase retention and graduation of students of color by 30% in two years.
Student success is predicated on the ability of educational leaders developing new strategies in identifying issues impeding the completion of a degree and retaining students who, if not for their circumstances, may have completed their degree. This notion involves the university holistically evaluating the implications of the current systems in place and enhancing them to implement plans that effectively retain and graduate minority students. The impact of supportive practices and pedagogies for underrepresented students is critical for retention and graduation rates where both the students and institutions would gain from newly implemented strategies that speak to the core mission and values of the university.
Description of the Program Evaluation
University of Delaware (UD) is a private-public research university located in Newark, Delaware, founded in 1743, making it one of the oldest universities in the nation and the largest university in the state of Delaware. More than 17,710 undergraduate students enrolled in 2020 at the main campus, (IRE, 2021a.) and by definition (SAGE, 2021) considered a predominantly white institution (PWI), hosting 68.4% White student population. Black or African American students are 5.7% of the population; Hispanic or Latino students are 8.8%; Asian students are 5.3%; Native American students make up 0.1%; International students are 5.3%; and Other students are 6.4% of the population. UD adopts a strategy that acknowledges the benefits of diversity for academic performance and fosters an inclusive approach by strengthening partnerships with other institutions, providing plentiful programs and opportunities for students, and creating a supportive learning environment. Concurrently, there are challenges associated with providing support for minority students, such as lack of resourcing, difficulties ensuring a holistic approach to the issue, and staff attitudes and practices regarding teaching minority students. The primary purpose of this evaluation is conducting an assessment of all racially and diversity-oriented university programs and assess the quality of their management, effectiveness, and respective evaluations in order to understand how these programs can be improved.
Evaluation Framework
The evaluation approach being used is a meta-assessment based on a framework developed by John Ory (1992). The evaluation framework suggests that assessment does not only measure student outcomes, but holistically encompasses the purposes and processes of the assessment of the specific programs and activities. The evaluation framework is based on four primary categories of utility (whether the evaluation is needed to serve practical information needs), accuracy (technical competency), proprietary (conforming to legal and ethical standards, and feasibility (whether the evaluation is realistic) (Ory, 1992). Therefore, taking the research topic at hand, all four of these apply. The evaluation holds a practical utility via necessity to facilitate greater retention and graduation rates for minorities. It is accurate, using key measurements and objectives. It is proprietary as the evaluation meets common standards used at university and meets legal and ethical standards. Finally, it is feasible, conducted at least once annually, the assessment of university programs is a realistic and manageable aspect. The evaluation framework provides 5-10 subcategories to each of the primary ones, with standards guiding the assessment with objective and encompassing methods to ensure the best and fair evaluation results.
In addition to the meta-assessment framework, an embedded evaluation framework can be used. The purpose of embedded evaluation is to essentially integrate the process of evaluation into program planning. The evaluation data would drive program improvement and be the basis for future decision-making for the specific program as to whether it should be expanded, improved, scaled down, or discontinued. Embedded evaluation is a ‘continuous’ assessment by focusing on how practical results meet program goals competently (Giancola, 2021). There is a relative freedom in creating and designing embedded evaluations with best fit the context and type of course/program being considered.
Stakeholder Engagement
The stakeholders included will all be internal as this is an internal university issue. The primary stakeholders are part of the Office of Institutional Equity, Diversity, and Inclusion. This includes the leader Interim Vice Provost, the offices of Division of Student Life, and the Equity and Inclusion team. Finally, there is the student-led organization the Center of Black Culture. The affected population themselves, minority students, are also involved as stakeholders themselves. The primary stakeholders to inform the program design will be the Interim Vice Provost, alongside with representatives from the two university offices. These stakeholders have tracked the issue for years, most likely have a wide range of data, and have a strong familiarity with existing programs. The strategies will be implemented by the two offices as well as the Center for Black Life which is directly invested in the improvement of black students’ opportunities. As can be seen in Appendix A, the stakeholders each have the specific interests and subsequently power to make changes as appropriate. The evaluation will be conducted by the author this paper, to ensure a level of integrity and avoid bias or bias confirmation. The data will first be shared with the Interim Vice Provost, and eventually to the other stakeholders to facilitate potential discussions and policy changes at the university. The data will likely be publicly available as an effort to be transparent in diversity efforts.
Description of Program
The goal of the project is to evaluate and change existing programs at the university. As discussed in the stakeholder section, a range of on-campus organizations and university entities are interested and involved in the final objectives. An assessment will be conducted of existing university programs. The two primary goals are to ensure diversity and inclusion are up to modern standards on campus, and to improve retention and graduation rates from racial/ethnic minorities. Other metrics that will be assessed are student satisfaction with the programs as well as their evaluations using a Likert scale on the effectiveness of these initiatives. The feedback generated will be first shared with the provost’s office and then distributed to respective university offices and student organizations. The two strategies for improvement are:
- Create a working and continuous assessment system with the university that would annually review data on the programs to evaluate their effectiveness, participation rates, cost-benefit analysis, and others in order to retain successful initiatives while discarding unsuccessful ones in favor of new developments.
- Provide students with appropriate and evidence-based support resources such as a tool kit of available university programs and organizations focusing on diversity and academic promotion, as well as potential strategies the students can employ in their day-to-day student life and education.
Affected Populations
The program is targeted primary at the university demographic of racial or ethnic minorities at the university. However, diversity and inclusion benefit the whole student body by making the university a safe environment and one where people of any type of ‘minority’ or ‘differences’ feel free to pursue academic opportunities and express themselves. Student bodies that have diversity bring significant value to students during and after their college career, by helping them learn about other cultures and differences, expand awareness with multiple perspectives, and build social skills (Shorelight Team, 2021). As for minority students, the program aims to ensure they are not facing any superficial barriers that may reflect discrimination from the university, provided equal opportunities, and also have the support systems in place to help navigate any specific cultural or socio-economic barriers. Students of all ages, majors, and programs are considered, but a specific attention is dedicated to undergraduates.
UD has the It is led by the Interim Vice Provost which serves as senior advisor to UD president on matters of diversity and collaborates with campus leaders on diversity and inclusion efforts. As part of that team are the Center for Black Culture formed in 1968, Division of Student Life, and Office of Equity and Inclusion (University of Delaware, 2021). Most of these have existed at the university since 1972 when the Educational Amendments of 1972 was federally passed into law, which maintains a range of policies on inclusion and diversity to receive federal funding. Therefore, the primary assessment and further changes will be conducted and discussed through this office.
Logic Model
The logic model will be used for this evaluation, as can be seen in Appendix E. The aim of the logic model is to demonstrate the shared relationships between inputs, resources, activities, outcomes, and impacts. Essentially, it trying to determine the relationship between program activities and intended effects. The context is that despite abundance of social and student support programs at UD, the retention and graduate rates for minority students, particularly Black and Hispanic, is the lowest. It is necessary to consider the programs to determine their effectiveness in creating socio-academic impacts. Some of the inputs are date on retention and graduation, students, current program activities, leadership buy/in, and program-policy restructure. The assumption is that minority students face risk factors and should have the same opportunity for retention and graduation as white students. External factors to consider are the policy and procedural changes, including outside the university, as well as limited resources and funding. Examining outputs, there are two categories of activities and participation. Activities include audit of student programs, program development and monitoring, toolkit for staff, and student surveys to understand their needs. Participating focuses on staff participating and training, reporting of program outcomes, and student input as stakeholders. Finally, there the outcomes – impact, broken down into short-, medium-, and long-term outcomes. While the short outcomes may include a better understanding and reporting, long-term objectives focus on those key goals of improving retention and graduation rates by at least 30% in two years. See Appendix E for greater detail.
Evaluation Design
The evaluation design will follow a non-experimental format, meaning there will be no comparison group. It will be an outcome evaluation and performance monitoring evaluation design. This means that it will be an ongoing evaluation at baseline and key milestones to provide continuous, real-time feedback on the program based on the embedded evaluation aspect. At the same time, an annual assessment of university programs will be conducted, that data then evaluated through the meta-assessment to determine how effectively the program is being assessed, its respective purpose, and the ability to meet program goals. However, it will also consider how well the project achieved its goals set in the beginning (RHIhub, n.d.). As can be seen in the Appendix F Evaluation Matrix, there are three levels of objectives that the program is attempting to achieve. Data will be collected using official university records and student surveys.
Implementation question: What assessments are best and most efficient to implement that can accurately monitor university programs on an annual basis? To what extent have programs been designed and implemented in ways that are aligned with UD’s diversity requirements and goals?
Outcome question: To what extent are university diversity and support programs for students of color effective in achieving their objectives and promoting high rates of retention and graduation among this demographic?
Description of Evidence and Data Sources
Both qualitative and quantitative data will be collected. For baseline data, the percentage of minority students present at the university, followed by the percentage of that sample participating or having used in the past one of the support programs for students. A short survey will be conducted determining opinions and satisfaction. This will occur in the short-term prior to implementation. Then there will the meta-assessment and evaluation integrated approach, during which each program will be evaluated using a qualitative approach but ensuring that each aspect meets the high standards of university diversity requirements and meets the goals that it is trying to achieve. This will occur in the medium term, annually after implementation. Finally, in the long-term, after at least 2-3 years, a long-term survey questionnaire will be sent to students who have participated regularly in these programs over the course of their college education, determining if participation rates have increased and their satisfaction with the programs. Meanwhile, university data will show any change in trends regarding graduation and retention rates after the staff toolkit has been developed and implemented with the efforts of improving these indicators.
The data collected will therefore include student survey data, including opinion and Likert Scale questions to determine satisfaction; program evaluation data; and university data for graduation and retention rates. The foundation good research and making evidence-based decisions relies on the trustworthiness of the data, which often depend on the instruments and methods used to collect the data. Generally, all the data should be trustworthy, as specific care will be used to design surveys and test for all three aspects which meet scientific rigorous standards. Reliability relies on consistency of measurements, while validity implies that the instruments measure what they are intended to measure. The biggest concern is with the surveys, both prior to implementation and after. They are meant to identify participation, satisfaction, and opinions on effectiveness of specific UD programs. It will be necessary to construct the surveys in a manner that clearly distinguishes each of these factors as to not have an overlap, such as opinions mixing in with data on satisfaction. A planned approach such as this would ensure validity, and if done correctly both times, reliability as well. As for the quantitative data, that is largely statistical, and will be subject to the same collection methods from databases and analyzed through the same measures.
Anticipated Findings
It is anticipated that the findings will demonstrate that only minor improvements are needed to the university programs. There are multiple programs and offices available with an opportunity for minority student to receive support both socially and academically, and the university actively promotes diversion and inclusivity on campus, with the large majority of staff and students supporting these efforts. Most likely, there will need to be minor changes to approaches and processes on how these programs operate, their goals, and practical utilization of services by minority students, particularly in academics. It is expected that some reform may be needed by developing and presenting an academic support and tutoring program for minority students through university offices and student organizations. In conclusion, diversity and inclusion efforts should improve while quantitative indicators such as retention and graduation rates may increase by a small percentage.
Communication Plan
Throughout the process described in this paper, it will be necessary to communicate with relevant stakeholders either for support, data collection, or evaluation purposes. The interim provost, who serves in the official capacity as head of diversity at UD, communication with them will be in the official capacity, fully pre-planned, and recorded. This contact will occur in written or face-to-face communication during strategic points in the project.
The next stakeholders are a student organization for students of color, and university offices responsible for diversity and inclusion along with other related aspects. They will be key partners in developing and implementing the project, with the goals of promoting diversity, offering opportunities to students of color, and ensuring UD programs meet appropriate standards. Frequent points of contact via email and period meetings will be beneficial to focus on solution-based approaches to campus-wide problems, while getting the perspectives of both student-led bodies and the university itself. Finally, minority students as a stakeholder will be contacted prior and after the implementation of the project and changes. These will be random encounters either at student events or on campus, asking to participate in the surveys. Contact will be either in-person or via e-mail, and the best approach would be to provide a brief context to what the project envisions and goals for the initiatives.
References
Giancola, S. P. (2021). Program evaluation: Embedding evaluation into program design and development. Thousand Oaks, CA: Sage Publications.
Grapin, S. L., & Pereiras, M. I. (2019). Supporting diverse students and faculty in higher education through multicultural organizational development. Training and Education in Professional Psychology, 13(4), 307–315
Green, S. L., & Wright, C. F. (2017). Retaining First Generation Underrepresented Minority Students: A Struggle for Higher Education. Journal of Education Research, 11(3), 323.
Institutional Research and Effectiveness (IRE 2021a.). (2020-2021) University of Delaware. Enrollment by Gender and IPEDS Race/Ethnicity.
Institutional Research and Effectiveness (IRE 2021b.). (2020-2021) University of Delaware. Graduation Rates for Entering First-time, Full-time, First-year Students by Gender IPEDS.
Institutional Research and Effectiveness (IRE 2021c.). (2020-2021) University of Delaware. Retention Rates for Entering First-time, Full-time, First-year Students by Gender IPEDS Race/Ethnicity.
SAGE (2021) Reference Encyclopedia. Predominantly White Institution.
Ory, J.C. (1992). Meta-assessment: Evaluating assessment activities. Research in Higher Education, 33(4), 467-481.
RHIhub. (n.d.).Evaluation design.
SAGE (2021) Reference Encyclopedia. Predominantly White Institution.
Shorelight Team. (2021). Why is cultural diversity important for colleges and universities?
University of Delaware. (2021a).Leadership. University of Delaware.
University of Delaware. (2021). Office of institutional equity, diversity, and inclusion.