Abstract
Objective: Automated text messaging interventions can effectively improve self-care and well-being and were used to support the public health outreach during the COVID pandemic in a large healthcare system. However, significant gaps exist in knowledge about patients’ preferences that may impact user receptivity, engagement, and effectiveness. This project qualitatively evaluated patients’ feedback for improving texting interventions and interest in future protocols.
Methods: We reviewed cross-sectional survey data from 1,134 patients receiving either the “Coronavirus Precautions” (“Precautions”) or the “Coping During COVID” (“Coping”) multi-week self-care text health subscriptions. Two team members independently and inductively coded responses for each health subscription. Fourteen categories emerged and were used to independently categorize all responses (inter-rater reliability 83.5%). Responses were organized around codes for types, specificity, and diversity of message content, frequency, timing, interactivity level, ability to connect, and technology of the messaging platform itself.
Findings/Results: Nine-hundred-five veterans (33.6% between 60-69 years old, 72.81% male) responded to the item assessing how a text health subscription could be improved. Prevalent codable findings across both text health subscriptions included a desire to manipulate message frequency (14.9% Precautions; 15.4% Coping) and the ability to have a more sophisticated or smarter interaction with the messages (15.3% Precautions; 14.8% Coping).
Conclusions: Patients offered suggestions that may impact receptivity and engagement of future text messaging interventions, particularly as they relate to text message outreach during a public health crisis. In addition, patients offered specific topics they would like to see in future text message health subscriptions. We discuss how the findings can be used to increase engagement in current automated text message interventions, as well as to develop post-pandemic public health interventions.
Introduction
Automated text messaging offers an innovative, cost-effective way to improve patients’ self-care and well-being. Interventions that consist solely of text messages have been found to effectively promote health self-management (Whittaker et al., 2019; Willcox et al., 2019). They have been found to increase patients’ health-related awareness and knowledge (Willcox et al., 2019) about a wide variety of issues (e.g., Sahin et al., 2021; Schwebel & Larimer, 2018) and help patients feel connected with their healthcare providers (Berrouiguet et al., 2016). Automated text messages allow for convenient information delivery and exchange, are accessible even in rural locales, and thus are suitable tools for public health outreach (Hall et al., 2015; Whittaker et al., 2019; Willcox et al, 2019). Additional research has shown that automated text messages are effective in decreasing rate of missed appointments (Clough and Casey, 2014; Tolonen et al., 2014), increasing compliance in completing vaccinations (Stockwell et al., 2014), and delivering educational, informational and motivational messages to clinical intervention (e.g., Celik et al., 2015; Kodama et al., 2016; Stockwell et al., 2014).
A growing percentage of US veterans also agree that text messaging is a desirable modality for promoting self-care (Erbes et al., 2014; McInnes et al., 2014; Saleem et al., 2020; Whealin et al., 2016). For example, a survey of 77 veterans with a mental health diagnosis found that 53% were interested in using technology for at least one type of healthcare-related communication (Miller et al., 2016). A survey of 106 homeless veterans found that most were interested in receiving mobile phone communications from their healthcare providers, with 93% interested in receiving mobile phone call or text message appointment reminders (McInnes et al., 2015). Among 290 veterans with spinal cord injuries and disorders, however, only 26% reported using text messaging (Hogan et al., 2016). Results of an implementation study of an automated text message intervention with 197 veterans provided similar results, including qualitative user feedback reports of messages being helpful and better adherence to medication (Yakovchenko, et al., 2019). Semi-structured interviews with VA healthcare staff and patients regarding their use of automated text messages demonstrated that the Annie text messages facilitated patient self-management and engagement in care (Yakovchenko et al., 2021).
When considering health technology applications it is important to consider the availability of the hardware and software to patients to provide access to all potential users. In the Department of Veterans Affairs, the automated text message platform, called Annie, uses Short-Message Service (SMS) text messaging to promote veteran self-care through interactive automated text prompts designed to track and monitor their health. Annie is available to any of the over nine million veterans enrolled in VA healthcare. An advantage of text messaging is that the technology is available to almost all types of cell phones, not just smart phones. While most veterans in the United States own smartphones (81.5% in a national sample of 3,078 US veterans; Jaworski et al., 2022), ownership among older veterans is lower (70% in one study of 77 veterans; Gould et al. 2020). Thus, it is important to offer options to meet all patient needs and preferences.
Despite widespread interest and growing use, very little research has examined patients’ preferences related to automated text message interventions. One study that surveyed 100 veterans who had attended an appointment at a single VA facility found that veterans expressed interest in mobile support to increase physical activity (80%), get better sleep (73%), change negative/self-critical thinking (72%), increase activities (72%), track mood/stress/anxiety/PTSD symptoms (67%), speak to a coach (66%), learn more about their mental health condition (65%), improve social skills (63%), receive medications reminders (61%), and connect with others who have similar problems (51%; Lipschitz et al., 2019). In another study, 28 older patients (mean age = 69.5, and 14 of whom were veterans) receiving antiplatelet medication participated in focus groups. The majority of patients viewed text message reminders favorably (Park et al., 2020). Most participants preferred text messages be sent from their own healthcare providers and be personalized to their condition or procedure. Most also desired a two-way texting interface to engage with the information within the text. Some participants saw the potential of text message fatigue that could result in ignored text reminders. Some also reported negative reactions to messages that praised them for adhering to their medication regimen, calling them “condescending” (Park et al., 2020).
Currently, gaps exist in knowledge about patients’ preferences for automated text messaging interventions to support patients during a public health crisis, and how we may use this knowledge to prepare for future crises. Additionally, most multi-site and national investigations of VA patients evaluating text messaging preferences employed a close-ended question format (e.g., Miller et al., 2016). Research employing qualitative methods will complement previous work. The aim of this study was to examine the receptivity, engagement, and effectiveness of this intervention based on qualitative feedback from users. This project also assessed patients’ suggestions for improving COVID texting health subscriptions and interest in future protocols.
Method
Sample
Every VA patient is eligible to use the Annie platform, and as a result, were eligible to be included in this study. Between March 2020 and March 2022, veteran enrollment in Annie grew from 6,500 to over 45,000, with users in all US states and territories. Users range from 18 to over 74 years old, with the largest percentage (29%) between 65-74 years (compared to the US veteran mean average age of 58 years, with most between 45-64 years; U.S. Department of Veterans Affairs, 2022).
Instrument
Survey items were developed by clinical expert group consensus (i.e., made up of an interdisciplinary team of clinical psychologists, other clinicians, and human factors professionals). Project items consisted of two open-ended (free text) response format questions: 1. “How could Annie’s messages be improved?” and 2. “Are there any specific topics you would like to see addressed in future Annie text message programs?”
Project Design
This project was a retrospective analysis of open-ended item survey response data captured as part of a national quality improvement initiative.
Procedure
We evaluated data collected from veterans enrolled in one or both of two population-scale automated, multi-week, self-care text interventions. The first, called Coronavirus Precautions (Precautions), was a 60-day text health subscription that delivered educational content on coronavirus safeguards, guided veterans in monitoring their clinical symptoms, and based on their responses, advised them to call their care team or a nurse triage line when appropriate (Saleem et al., 2020). Educational tips (e.g., “Annie again. If you are ill, use a separate bedroom and bathroom if possible”) were sent Monday, Wednesday, and Friday, and wellness questions (“Annie here. Are you feeling well today?”) were sent Monday, Wednesday, Friday, and Sunday. Beginning March 2020, the protocol was available on the VA’s text messaging platform “Annie.”
The second text intervention, called “Coping During COVID” (Coping) was a 60-day health subscription that sent one of four rotating message prompts on Tuesdays, Thursdays, and Saturdays (Whealin, Saleem, Vetter, Roth, & Herout, 2021). In these messages, “Annie” prompts veterans to take steps every day to prevent stress and states that veterans can “type TIP YES to request a coping tip now or whenever you want one.” Veterans could receive one of 60 motivational/educational text messages designed to encourage proactive efforts to manage COVID-19-related stress, such as, “Set a new goal for something you’d like to accomplish in the month ahead. What would you like to change in your life?” and “Dwelling on negative thoughts is not good for your health. Balance out negative news by focusing on positive thoughts.” The health subscription became available in May 2020. Any veteran registered as a VA patient could self-enroll for free in either/both health subscription(s) or be assigned by a VA staff member.
To collect data, Precautions and Coping subscribers were texted a Survey Monkey Internet link which hosted the evaluation tool. Approval for the project was completed through the Clinical Coordination Cell, a function of the VHA Emergency Management Coordination Cell.
The VA Annie Platform
The Precautions and Coping health subscriptions are part of the VA Office of Connected Care (OCC) automated SMS tool ‘Annie’ The platform allows for delivery of automated SMS (short message service) text messages to practically all cell phones. Typical texting plan charges may apply.
Data Analysis
We reviewed cross-sectional survey data from 1,134 veterans receiving either the Precautions or the Coping During COVID subscriptions. Two team members independently and inductively coded the first 50 responses for each participant. Fourteen categories emerged and were used to independently categorize all responses (inter-rater reliability 83.5%).
Results
Patients’ suggestions for improving the Precautions and the Coping text messaging health subscriptions, the codes used to categorize them, and example veteran responses are summarized in Table 1. Patients’ suggestions for other Health subscriptions are listed in Table 2.
Suggestions for Improvement
Content
A majority of responses (collectively) related to the content of the text messages. These responses were further sub-categorized into content-specific areas.
Information
Veterans expressed wanting to receive specific types of information through the text messages, particularly with the Precautions health subscription. The information sought through Precautions was quite varied. For instance and included a desire for tailored information about COVID-19 for the specific area they lived in such as infection rates and testing sites. Other examples of information sought included more detail about the COVID-19 virus and symptoms. For the Coping health subscription, some responses also related to more information about COVID-19. However, other responses expressed a desire for more information about coping activities for themselves and others.
Specificity
In addition to feedback types of questions desired, veterans also expressed a need for greater specificity of the current questions. A couple veterans using the Coping health subscription suggested having messages that were tailored to gender and age.
Other
Responses in the ‘Other’ sub-category included a variety of feedback more broadly related to other content suggestions.
Diversity
For both health subscriptions, veterans expressed a desire for a more variety of messages and less repetition of the current ones.
Recommendations/tips
Veterans expressed a desire for more recommendations and tips. While the Precautions health subscription is intentionally designed with a decision tree with wellness check questions to guide the veteran on whether they need to call the VA triage nurse or care team, the Coping health subscription is more focused on suggesting activities that may help manage COVID-related stressors. Even so, veterans expressed a want for even more suggestions.
Questions
Some veterans expressed a desire for more specific and pertinent questions from the incoming text messages from both health subscriptions. Others using Precautions wanted “less” questions and “wider” questions. One veteran using Coping wanted a scalar response option for questions; e.g., “[from] 1 – 5 how well is your mental energy?”
Mental health
For both the Precautions and Coping health subscriptions, some veterans wanted content more focused on mental health.
Interaction
Veterans wanted more interactivity with Annie’s text messages for both health subscriptions. The types of responses allowed by Precautions health subscription are viewed as too rigid by some veterans.
Frequency
Veterans using both protocols wanted the ability to adjust the frequency of the incoming text messages. Further, there was variation in the direction of desired frequency (less vs. more) for both protocols. However, most receiving the Precautions health subscription wanted fewer messages. In comparison, most responses from the Coping health subscription related to frequency wanted more frequent messages rather than less.
Technology
Some suggested improvements for both health subscriptions related to the usability or capabilities of the technology of the messaging platform itself, rather than the actual text messages, such as making Annie into an app, rather than a simple text messaging service, that is more usable. For the Coping During COVID subscription specifically, several veterans wanted tips to be automatically sent rather than having to first request one.
Connect with VA
Veterans expressed a greater desire to be able to connect with the VA through both health subscriptions. For example, by including phone numbers in the text messages that one could call to speak with a VA staff member, or by automatically triggering a call from a VA care provider based on the veteran’s responses to the messages, or by linking to other VA systems and relevant VA Web sites.
Time
For Precautions, several veterans wanted the timing of the messages to arrive earlier in the day. However, this was a technical error from the VA Annie team, which erroneously sent messages after 10:00pm. This error was later fixed prior to the initiation of the Coping health subscription; no veterans responded with feedback about the timing of the Coping messages.
Positive/Other
A final category of suggested improvements was labelled as ‘Other’ as the feedback did not fit our established coding scheme (e.g., “offer podcast”). We also included positive affirmation type responses in this category.
Suggestions for other Health subscriptions
Veterans who participated in the Coping health subscription were also asked ‘Are there any specific topics you would like to see addressed in future Annie text message programs?’ Reponses were coded only once, based upon the most prominent theme they represented. Topic categories requested by more than one veteran, with example responses, are summarized in Table 2.
Discussion
This project evaluated preferences for automated text messaging interventions among a national sample of patients receiving COVID text messaging health subscriptions offered by the Department of Veterans Affairs. Our open-ended format allowed patients to describe preferences that were most relevant to them.
Of issues noted, the most-frequently mentioned addressed the content of the messages. For example, patients who received the Precautions subscription identified specific, and often up-to-date, information about COVID-19 for the specific area they lived in (such as infection rates, testing sites and current information about the COVID-19 virus and symptoms). Those received the Coping health subscription most often expressed a desire for more information about as additional coping skills, as well as coping activities for family members and others in their lives.
A second common request was for more interactivity to either tailor the subscription to individual needs, or to provide a more natural chat bot-type, back and forth conversation. Message tailoring refers to the practice of designing messages at the individual level (Kreuter et al., 2000). Users ten to perceive a more customized message as more personally relevant, and enhanced personal relevance can promotes greater attention and resulting behavior change (Noar et al., 2009).
Moreover, the use of more sophisticated text-and voice-based technologies, often referred to as intelligent virtual agents (IVAs), has become increasingly widespread. This is true not only for younger adults, but for older adults as well (Blocker et al., 2020; Harrington et al., 2020; Koon, et al., 2020; Korok et al., 2020). Compared to simple text messaging subscriptions, IVAs offer more sophisticated responses tailored to the needs of the user and more interactivity via AI-based conversational abilities (e.g., Apple’s Siri and Amazon’s Alexa) (Blocker et al., 2020; Harrington et al., 2020; Koon, et al., 2020; Korok et al., 2020). The Annie platform was specifically chosen, in part, to include those veterans who still prefer earlier phone models (including flip phones), rather than smart phones. However, as time goes on, VA can transition to new technology reflective of the growing needs and level of tech-savviness of the VA-patient population, while simultaneously accommodating veterans with older model phones.
Another common request was for the VA to address content that focuses on mental health needs, which was repeated in our follow-up item that inquired about specific topics patients would like to see addressed in future Annie text message programs. Responses to the latter item included interest in mental health techniques (e.g., breathing exercises, staying focused on a task, defusing anger stressors, and increasing positive thoughts) as well as those addressing particular issues (e.g., anxiety, PTSD, grief, depressive symptoms). Another broad category of frequently requested topics were those addressing physical health and well-being, including weight management and exercise.
Our findings are consistent with previous investigations of VA patients in which they expressed interest in eHealth tools designed to help them increase exercise (75.8%), improve sleep (73.2%), change negative thinking (70.5%), and increase involvement in other types of activities (67.1%; Lipschitz et al., 2019). Of notes, Annie already does have subscriptions that address many of these topics. A study of 140 patients in one VA facility found that only 42.5% and 20.4% of patients had heard of and used the current VA/DoD mental health apps, respectively (Reger et al., 2021). Additionally, lack of awareness was the most common barrier to app use, endorsed by over two thirds of patients (65.7%). The present findings suggest that VA patients would like to see these subscriptions that already exist, speaking to the need to continue seek new ways to disseminate information about their existence.
Other factors noted to improve adoption include provider endorsement, greater publicity of which tools are efficacious, and clear information about privacy of information.
Lastly, patients in the present project also request information to support family relationships and couples counseling tips.
Limitations
Note that our findings are limited to a VA patient population, which tends to older and with many more males than other populations. Future research should evaluate patient preferences are different across demographic subgroups (e.g., gender, age, socio-economic status) to best meet the needs of different segments of the patient population both within and outside of VA. Additionally, human factors studies of VA patients could help improve the interactivity of patients’ experiences receiving and responding to the automatic messages, and enable a better understand the interface of the patient with their device.
Conclusions and Clinical Implications
This qualitative project complements previous work (e.g., Miller et al., 2016) to provide information about the patients perspective, receptivity, engagement, and effectiveness of for improving COVID texting health subscriptions and interest in future protocols.
Cell phones are omnipresent, and despite their simplicity, automatic text message subscriptions can be a valuable communication tool for supporting self-care, especially during widespread pandemics and disaster.
Table 1. Coding Dictionary, Summary Counts and Examples of Patients’ suggestions
Note: Percentages are based on codable data. P: Precautions health subscription, C: Coping protocol.
Table 2. Patients’ Suggested Subject Areas for Future Texting Protocols
Note: Coded topics were those suggested by more than one patient. Percentages are based on codable data
References
Berrouiguet, S., Baca-García, E., Brandt, S., Walter, M., & Courtet, P. (2016). Fundamentals for future mobile-health (mHealth): a systematic review of mobile phone and web-based text messaging in mental health.Journal of Medical Internet Research, 18(6). Web.
Blocker, K. A., Kadylak, T., Koon, L. M., Kovac, C. E., & Rogers, W. A. (2020). Digital home assistants and aging: Initial perspectives from novice older adult users.Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 64(1), 1367–1371. Web.
Celik, S., Cosansu, G., Erdogan, S., Kahraman, A., Isik, S., Bayrak, G., Bektas, B., Olgun,. N. (2014). Using mobile phone text messages to improve insulin injection technique and glycaemic control in patients with diabetes mellitus: a multi-centre study in Turkey. J. Clin. Nurs. 24(11−12), 1525–1533
Clough, B.A., & Casey, L.M., (2014). Using SMS reminders in psychology clinics: a cautionary tale. Behav. Cogn. Psychother. 42(3), 257–268
Erbes, C. R., Stinson, R., Kuhn, E., Polusny, M., Urban, J., Hoffman, J., Ruzek, J. I., Stepnowsky, C., & Thorp, S. R. (2014). Access, utilization, and interest in mhealth applications among veterans receiving outpatient care for PTSD.Military Medicine, 179(11), 1218–1222. Web.
Gould, C. E., Loup, J., Kuhn, E., Beaudreau, S. A., Ma, F., Goldstein, M. K., Wetherell, J. L., Zapata, A., Choe, P., & O’Hara, R. (2020). Technology use and preferences for mental health self-management interventions among older veterans.International Journal of Geriatric Psychiatry, 35(3), 321–330. Web.
Hall, A., Cole-Lewis, H., & Bernhardt, J. (2015). Mobile text messaging for health: a systematic review of reviews.Annual Review of Public Health, 36(1), 393–415. Web.
Harrington, C. N., Koon, L. M., & Rogers, W. A. (2020). Design of health information and communication technologies for older adults. In A. Sethumadhavan & F. Sasangohar (Eds.), Design for Health (pp. 341–363). Academic Press. Web.
Hogan, T. P., Hill, J. N., Locatelli, S. M., Weaver, F. M., Thomas, F. P., Nazi, K. M., Goldstein, B., & Smith, B. M. (2016). Health information seeking and technology use among veterans with spinal cord injuries and disorders.M & R: The Journal of Injury, Function, and Rehabilitation, 8(2), 123–130. Web.
Jaworski, B. K., Taylor, K., Ramsey, K. M., Heinz, A. J., Steinmetz, S., Owen, J. E., Tsai, J., & Pietrzak, R. H. (2022). Predicting uptake of the COVID Coach app among US military veterans: funnel analysis using a probability-based panel. JMIR Ment Health, 9(4), e36217.
Kodama, T., Syouji, H., Takaki, S., Fujimoto, H., Ishikawa, S., Fukutake, M., Taira, M., Hashimoto, T. (2016). Text messaging for psychiatric outpatients: effect on help-seeking and self-harming behaviors. J. Psychosoc. Nurs. Ment. Health Serv. 54(4), 31–37.
Koon, L. M., McGlynn, S. A., Blocker, K. A., & Rogers, W. A. (2020). Perceptions of digital assistants from early adopters aged 55+. Ergonomics in Design, 28(1), 16–23. Web.
Korok, S., Sarcar, S., Pradhan, A., McNaney, R., Sayago, S., Chattopadhyay, D., & Joshi, A. (2020). Challenges and opportunities of leveraging intelligent conversational assistant to improve the well-being of older adults. In Extended abstracts of the 2020 CHI conference on human factors in computing systems (pp. 1–4). Association for Computing Machinery. Web.
Lipschitz, J., Miller, C. J., Hogan, T. P., Burdick, K. E., Lippin-Foster, R., Simon, S. R., & Burgess, J. (2019). Adoption of mobile apps for depression and anxiety: Cross-sectional survey study on patient interest and barriers to engagement.JMIR Mental Health, 6(1). Web.
McInnes, D. K., Sawh, L., Petrakis, B. A., Rao, S. R., Shimada, S. L., Eyrich-Garg, K. M., Gifford, A. L., Anaya, H. D., & Smelson, D. A. (2014). The potential for health-related uses of mobile phones and internet with homeless veterans: Results from a multisite survey.Telemedicine and e-Health, 20(9), 801–809. Web.
Miller, C. J., McInnes, D. K., Stolzmann, K., & Bauer, M. S. (2016). Interest in use of technology for healthcare among veterans receiving treatment for mental health.Telemedicine and e-Health, 22(10), 847–854. Web.
Noar, S. M., Harrington, N. G., & Aldrich, R. S. (2009). The role of message tailoring in the development of persuasive health communication messages. In C. S. Beck (Ed.) Communication Yearbook 33 (pp. 73 – 133). Lawrence Erlbaum Associates, Inc.
Park, L. G., Ng, F., K Shim, J., Elnaggar, A., & Villero, O. (2020). Perceptions and experiences of using mobile technology for medication adherence among older adults with coronary heart disease: A qualitative study.Digital Health, 6, 205520762092684. Web.
Reger, G. M., Harned, M., Stevens, E. S., Porter, S., Nguyen, J., & Norr, A. M. (2021). Mobile applications may be the future of veteran mental health support but do veterans know yet? A survey of app knowledge and use.Psychological Services. Advance online publication. Web.
Sahin, C., Courtney, K. L., Naylor, P. J., & Rhodes, R. E. (2020). Patients’ evaluations of mobile text messaging studies for type 2 diabetes management: a systematic review and a meta-synthesis.Journal of Technology in Behavioral Science, 6(1), 54–73. Web.
Saleem, J. J., Read, J. M., Loehr, B. M., Frisbee, K. L., Wilck, N. R., Murphy, J. J., Vetter, B. M., & Herout, J. (2020). Veterans’ response to an automated text messaging protocol during the COVID-19 pandemic.Journal of the American Medical Informatics Association, 27(8), 1300–1305. Web.
Schwebel, F. J., & Larimer, M. E. (2018). Using text message reminders in health care services: a narrative literature review.Internet Interventions, 13, 82–104. Web.
Stockwell, M.S., Westhoff, C., Kharbanda, E.O., Vargas, C.Y., Camargo, S., Vawdrey, D.K.,Castaño, P.M. (2014). Influenza vaccine text message reminders for urban, low-income pregnant women: a randomized controlled trial. Am. J. Public Health, 104(1), e7–e12. Web.
Tolonen, H., Aistrich, A., & Borodulin, K. (2014). Increasing health examination survey participation rates by SMS reminders and flexible examination times. Scand. J. Public Health, 42(7), 712–717.
U.S. Department of Veterans Affairs (2022). VA National Center for Veterans Analysis and Statistics. Washington DC.
Whealin, J. M., Jenchura, E. C., Wong, A. C., & Zulman, D. M. (2016). How veterans with post-traumatic stress disorder and comorbid health conditions utilize eHealth to manage their health care needs: A mixed-methods analysis.Journal of Medical Internet Research, 18(10). Web.
Whealin, J. M., Saleem, J. J., Vetter, B., Roth, J., & Herout, J. (2021). Development and cross-sectional evaluation of a text message protocol to support mental health well-being.Psychological Services. Web.
Whittaker, R., McRobbie, H., Bullen, C., Rodgers, A., Gu, Y., & Dobson, R. (2019). Mobile phone text messaging and app-based interventions for Smoking Cessation. Cochrane Database of Systematic Reviews. Web.
Willcox, J. C., Dobson, R., & Whittaker, R. (2019). Old-fashioned technology in the era of “Bling”: Is there a future for text messaging in health care?Journal of Medical Internet Research, 21(12). Web.
Yakovchenko, V., Hogan, T. P., Houston, T. K., Richardson, L., Lipschitz, J., Petrakis, B. A., Gillespie, C., & McInnes, D. K. (2019). Automated text messaging with patients in Department of Veterans Affairs specialty clinics: cluster randomized trial.Journal of Medical Internet Research, 21(8), e14750. Web.
Yakovchenko, V., McInnes, D. K., Petrakis, B. A., Gillespie, C., Lipschitz, J. M., McCullough, M. B., Richardson, L., Vetter, B., & Hogan, T. P. (2021). Implementing automated text messaging for patient self-management in the Veterans Health Administration: qualitative study applying the nonadoption, abandonment, scale-up, spread, and sustainability framework.Journal of Medical Internet Research: mHealth and uHealth, 9(11), e31037. Web.