The study “Mixed-method process evaluation of the Welsh National Exercise Referral Scheme” by Moore analyzes the ways of countering insufficient activity in adults. In particular, the research concerns exercise referral schemes (ERS), involving visits to leisure facilities and programs with instructors, and National Exercise Referral Scheme (NERS), introduced in 12 Welsh local health boards. Moore et al. (2013) examine randomized controlled trials (RCTs) conducted earlier and present disappointing results of these practices (p. 476). The study utilizes a mixed-method process of evaluation.
The Results of a Pragmatic RCT
The findings show that motivational interviewing (MI), an essential counseling technique in NERS, was not adequately delivered due to the supervisors’ and program developers’ insufficient qualifications and lack of training. Furthermore, the study shows that only a minority of patients were ready to continue the program. At the same time, a third of participants did not visit even the first appointment, and the rest did not complete referral exercises (Moore et al., 2013, p. 478). Among patients referred for mental health reasons, the adherence rate was higher with elderly people, while women were more likely to enter the program but did not adhere to it. Contradictory results show that NERS uptake was generally reduced in deprived areas or while attending patients with lower education. The scheme effectiveness has a non-significant higher impact on the physical activity of women and is not patterned by age or financial status. The research by Moore et al. (2013) suggests that undervalued qualitative methods may explain patients’ motivation to visit an ERS (p. 478). The existing studies show that older patients attend these programs to improve their fitness and social connectedness. The ERS highly depends on the instructor and collective exercises, which help in the assimilation of the patients.
The Aims of the Paper
The study’s main aims are to conduct a quantitative analysis of the fidelity, patterning, adherence level of NERS patients. It also attempts to present the qualitative evaluation of social patterning and explain patients’ motivation to follow physical activity programs. The NERS intervention referred patients who led a sedentary way of life and had at least one other health condition to local facilities, the choice of which was based on requests of professional advice. Patients were offered a 16 weeks program focused on group-based exercises. Moreover, the program offered various activities and class times. Patients were encouraged to have two weekly sessions, and professionals were required to contact those who dropped out. Further telephone contact was sustained to monitor the patients’ progress with a final check-up at 12 months. The NERS intervention did not achieve prominent results as the patients’ motivation did not reach the necessary levels because of the professionals’ poor qualifications. However, the influence of the physical activity provided by the program is definitely positive and should not be disregarded.
A Mixed Methods Process Evaluation
Based on its aims, the study utilizes mixed methods of evaluation. The quantitative approach involved the assessment of attendance levels. Patients’ profiles were examined with the questionnaires, presenting the baseline data of age and sex. They were later divided into four groups based on the level of their physical activity. The qualitative method assessed patients from different regions and levels of deprivation. Interviews were chosen as the main instrument in order to research patients’ motivation to attend the program, explore their opinions and ideas. The approach involved both group and individual interviews based on the number of attendees and types of sessions.
The mixed approach to evaluation allowed researchers to combine the advantages of both methods. The quantitative analysis provided the study with a wide range of data, reaching a higher sample size and increasing the chances of accurate results. By dividing patients into various groups on the basis of questionnaires and assessing both entering and completion rate, the specialists lowered the margin of error. However, the quantitative analysis alone would not be enough to reach the aims of the study and explain patients’ motivation and this social phenomenon in general. Qualitative research was necessary to describe the impact of NERS on examinees. Interviews based on the opportunistic approach in the choice of the examinees reduced the chance of bias, presenting an honest perspective on the matter. Furthermore, qualitative findings were important to assure the deliverance of the planned activities.
Dose Delivered
The delivered dose was defined and measured in different ways. The consultations were recorded on tape, which allowed to measuring their mean length and averaged 34.8 minutes (Moore et al., 2013, p. 484). The program length was longer than intended. Visits on the first day and exit day were recorded, and it took almost 19 weeks to finish the scheme with the recommended 12. The number of classes per center varied, reaching 4.3 on average. The data were gathered from the interview with the exercise professionals. Discounts were not provided ubiquitously and were mostly accessible by the completers and limited to £1 rate, based on the information from coordinator interviews. Post scheme discounts were provided by most of the LHB areas as stated by coordinators. Withdrawal of support level at 16 weeks reached 36.8 percent, resulting from professionals preventing class access after the expiration date. The discussed processes were the main components of the delivered dose.
The Impacts and Caveats of the ERS
The study produced several findings concerning the potential benefits of physical activity and mental health changes provided with the ERS. First of all, there is inconsistency within the programs and classes of the NERS. While the core of the supervised activities remained the same, the time range of classes was mostly limited to weekends, evenings, and off-hours. Findings by Moore et al. show that consultations varied from 12 minutes to an hour. It means that chances to provide high-quality one-on-one consultations were reduced. The necessary components might have been omitted, preventing patients from receiving the full impact of MI. This might be one of the reasons why the participants’ motivation did not change to a great extent after the procedures. The possible solutions may include separation of consultations and activities, creation of individual schedules, reduction of the participants’ number to better attend to each person.
One of the findings discovered that MI was not properly delivered. Researchers suggest that the patients’ reasons for completing the NERS were self-determined (Moore et al., 2013, p. 494). It proves the professionals’ inability to create an appealing image of the program and poorly delivered activities. The problem may lie in the personnel’s lack of proper qualification or lack of proper attention to each individual’s needs. While the patients face some motivational barriers, it is the job of the specialists to provide proper support and encourage them to continue treatment. According to Moore et al. (2013), many patients reported anxiety concerning the loss of professional support later in the program (p. 495). The above-mentioned incapacity to arrange full-time consultations might have played its role. If the professionals had more time to attend to patients’ fears, it could have been possible to engage more people in the NERS. The possible solutions may include the improvement of the personnel’s communication techniques by providing special training and improve their qualifications or by allocating more people for consultations and activities. However, both solutions require increased funding which might have been impossible under those circumstances.
The recommended number of classes was also rarely achieved, and patients could not fully engage in the NERS. Special discounts were provided for the patients to improve the situation, which was still accessible after the time period of 12 months. The additional support from the professionals hindered the results of the study as the benefits of the NERS could not be examined after the expiry of these discounts. The only possible suggestion is to conduct an additional study or provide a survey after the supported withdrawal to complement the current research. However, it is possible that the support affects only the patients’ motivation, while the results of physical activity on general being and mental health remain the same. The researchers might have also included additional questions in their interviews or survey concerning the conditions under which patients agree to continue the NERS and how discounts affect them. Lastly, there is an issue of a small range of patients, with the predominance of older women, preventing younger patients from successful assimilation. It is possible to divide classes based on age or sex or to provide emotional support and consultations on the matter of psychological barriers.
The SERS Implementation
An implementation of the already existing scheme requires formative evaluation to examine the potential areas of concern such as adaptation, context, and possible response to change. Formative evaluation helps to identify at early stages the results of the desired outcomes and refine implementation strategies (Elwy et al., 2019, p. 2). Before the national roll-out, it is important to inform the management group on program quality and possible improvements in order to succeed and avoid the mistakes from the NERS.
This study should also include the mixed approach for the research method. The combination of two methods helps to clear any contradictions that may arise between qualitative and quantitative findings. This approach is highly flexible and may be used as a basis for the future stages of the SERS program, preventing the necessity to conduct multiple studies. The interpretive description may be relevant for the study development by analyzing the first-hand patients’ experiences with chronic illnesses and used to refine possible interventions. According to Teodoro et al. (2018), this method helps to understand better participants’ preferences for the behavior of physicians, attitudes to physical activity practice, professionals’ perspectives for work in the local facilities (p. 4). The quantitative experimental approach in the study determines the relationship between patients’ health before and after the treatment, exploring the possible results of the SERS and establishing causality. The type of integration between the two methods should rely on their connection, as it provides the understanding of participants’ motivation and feelings of social support (Kaur et al., 2019, p. 2). Study design should be of convergent type when both methods are applied concurrently, which allows to implement changes in the case of need.
Data for formative evaluation may be gathered from various sources. As the SERS is focused on the groups with chronic illnesses, it is important to take this factor into consideration. The primary targets should involve patients found at higher risk and who lack proper social support. De Maria et al. (2018) suggest that the prevalent chronic diseases include people with heart failure (HF), chronic obstructive pulmonary disease (COPD), and diabetes mellitus (DM), forming three groups who do not receive proper attention (p. 308). These patients are susceptible to high risks, require proper care, and should be attended in the first place. Other possible stakeholder groups involve Health Promotion Board professionals, representing care providers, who provide consultation, and family members as the main social support in the life of the chronic illness patients. By focusing the research’s attention on these three stakeholder groups, it is possible to find and eliminate the downsides of the SERS program that may arise both inside and outside of the facilities.
The qualitative part of the research should employ structured interviews as the main data collection method regarding patients’ physical and mental condition as well as motivation before and after the program. The interviews should be conducted both with the patients and their family members to find out any critical deteriorations that may happen in the facility or at home. The quantitative part of the research relies on surveys such as the Patient Health Questionnaire and provides a basic understanding of the case severity, serving as a criterion for the approval for the scheme (Lervik et al., 2020, p. 4). HPB professionals may also undertake surveys and interviews regarding their training and qualification. The NERS example has shown that specialists play a major role in the program, and their lack of skills may endanger people with serious health conditions.
For the formative evaluation, the participants should be selected according to the combination of stratified random and non-probability judgmental sampling methods. It means that patients from different social classes with different demographic variables (age, gender, marital status) should be allowed to enter the program. However, to reduce the number of participants for the initial stage and provide better control for any deteriorations, the study should focus only on those patients who have high levels of anxiety and depression due to their condition. The entered participants should undertake the survey, and further groups will be formed based on the discussed criterion. In this case, varied but small groups will be formed, providing valuable information for the future adaptation of the system in Singapore. The participants for the family members’ interviews may be chosen with simple random sampling as they do not play a major role in the study.
References
De Maria, M., Vellone, E., Durante, A., Biagioli, V., & Matarese, M. (2018). Psychometrics evaluation of the Multidimensional Scale of Perceived Social Support (MSPSS) in people with chronic disease.Annali del l’Istituto Superiore di Sanita, 54(4), 308-315. Web.
Elwy, A. R., Wasan, A. D., Gillman, A. G., Johnston, K. L., Dodds, N., McFarland, C., & Greco, C. M. (2020). Using formative evaluation methods to improve clinical implementation efforts: Description and an example.Psychiatry Research, 283, 1-6. Web.
Kaur, N., Vedel, I., El Sherif, R., & Pluye, P. (2019). Practical mixed methods strategies used to integrate qualitative and quantitative methods in community-based primary health care research.Family Practice, 36(5), 666-671. Web.
Lervik, L. V., Knapstad, M., & Smith, O. R. F. (2020). Process evaluation of Prompt Mental Health Care (PMHC): The Norwegian version of improving access to psychological therapies.BMC Health Services Research, 20, 1-17. Web.
Moore, F.G., Raisanen, L., Moore, L., Ud Din, N., & Murphy, S. (2013). Mixed-method process evaluation of the Welsh National Exercise Referral Scheme.Health Education, 113(6), 476-501. Web.
Teodoro, I. P. P., Rebouças, V. D. C. F., Thorne, S. E., Souza, N. K. M. D., Brito, L. S. A. D., & Alencar, A. M. P. G. (2018). Interpretive description: A viable methodological approach for nursing research. Escola Anna Nery, 22(3), 1-8. Web.