CareMessage Text Usage Increases Appointment Adherence in a

Journal of Student-Run Clinics

Original Study

journalsrc.org | J Stud Run Clin 8;1 | 1

CareMessage Text Usage Increases Appointment

Adherence in a Student-Run Free Clinic

Evan J Chen

; Peter Hoang

; Katherine Garcia

; Nicholas Richwagen

; Magdalena Pasarica, MD, PhD

University of Central Florida College of Medicine, Orlando, Florida, USA

Corresponding Author: Magdalena Pasarica MD, PhD; email: magdalena.p[email protected]u

Published: April 21, 2022

Abstract

Introduction: Keeping Neighbors in Good Health Through Service Clinic is a student-run, multidisci-

plinary free clinic dedicated to providing healthcare to uninsured, low-income patients in Central Flor-

ida. A quality improvement project was performed to decrease the rate of patient no-shows via alter-

ing the patient appointment communication method.

Methods: A needs assessment was performed from June 2017 to March 2018 (n=104 patients). The

intervention was implemented over an 18-month period, from April 2018 to September 2019 (n=243

patients). The intervention changed the communication method with patients such that they re-

ceived appointment reminder texts two days before their appointments and responded to confirm or

cancel their appointments. Phone calls were used for rescheduling, confirming cancellations, or con-

tacting patients who had not responded to the CareMessage text. The Mann-Whitney U test was uti-

lized for comparison of appointment no-shows, number of phone calls made, and number of patients

scheduled per clinic between pre- and post-intervention groups.

Results: The needs assessment showed that the average number of no-shows was 0.80 per clinic,

while the average number of no-shows post-intervention was 0.26 per clinic (p=0.040). The average

number of patients scheduled per clinic pre- and post-intervention was 6.90 and 7.84 (p=0.370), re-

spectively. The average number of patients seen per clinic pre- and post-intervention was 5.93 and

6.74 (p=0.640), respectively. The average number of phone calls made weekly was 9.13 per clinic pre-

intervention and 3.23 post-intervention (p<0.001).

Conclusion: CareMessage is effective in reducing appointment no-shows and the number of follow-

up phone calls made for appointment reminders. This intervention is important for increasing patient

continuity of care, access to care, and clinic efficiency.

Introduction

High numbers of patient appointment no-

shows adversely impact clinic efficiency, produc-

tivity, and patient care. The increasing use of mo-

bile technology, especially smartphones, has of-

fered physicians a rapid and automated ap-

proach to patient communication. Multiple stud-

ies have demonstrated that text messaging re-

minder services in healthcare can significantly

decrease the number of clinic appointment no-

shows.

1-6

For instance, a meta-analysis reviewed a

series of 18 randomized controlled trials and ob-

servational studies to assess patient attendance

rates. They concluded that the use of text mes-

sage reminders increased the likelihood of pa-

tient attendance by 50% when compared to con-

trol (no reminders).

When specifically examining

underserved communities, a 2016 study explored

the impact of text message reminders on ap-

pointment adherence in a community health

center. They concluded that patients who re-

ceived text message reminders had a lower pro-

portion of appointment no-shows compared to

those who did not.

However, there are only few

studies that assessed the use and impact of mo-

bile technology and text message reminder sys-

tems in free clinics serving low-income

Journal of Student-Run Clinics | CareMessage Text Usage Increases Appointment Adherence in a Student-Run Free Clinic

journalsrc.org | J Stud Run Clin 8;1 | 2

populations.

7-9

We sought to improve patient appointment

adherence in order to improve clinic efficiency at

the Keeping Neighbors in Good Health Through

Service (KNIGHTS) Clinic, a student-run multidis-

ciplinary free clinic that provides healthcare to

uninsured patients living 200% below the federal

poverty level in Central Florida. In the past, medi-

cal student care coordinators at KNIGHTS clinic

confirmed patient appointments through direct

phone calls. This mode of communication proved

to be ineffective due to discrepancies between

care coordinator and patient schedules, which of-

ten resulted in numerous patient no-shows and

last-minute cancellations. CareMessage (CM)

(2012, San Francisco, California) is a non-profit or-

ganization that developed a messaging platform

in 2012 aimed at improving patient-provider

communication in underserved healthcare set-

tings. The CM platform allowed student care co-

ordinators to reach patients with the Health In-

surance Portability and Accountability Act pro-

tection on any mobile device in English or Span-

ish languages. This manuscript aims to evaluate

the efficacy of the CM platform in improving pa-

tient scheduling and appointment confirmation

for underserved populations in Central Florida.

Methods

KNIGHTS Clinic operates at Grace Medical

Home (GMH), an establishment that provides

comprehensive care for underserved patients in

Central Florida. GMH provides the facility as well

as necessary medical and laboratory supplies for

the student-run free clinic. A total of 347 low-in-

come, uninsured patients with scheduled pri-

mary care or specialty care appointments at

KNIGHTS Clinic were involved in this quality im-

provement study. Clinic sessions were held on a

bimonthly basis. During each clinic, up to 10 ap-

pointments could be scheduled. The study inter-

vention involved full implementation of the CM

text message system for confirmation of patient

appointments. Of the total study population, 104

patients were studied prior to intervention over a

10-month needs assessment period (June 2017-

March 2018), and 243 were studied post-interven-

tion over an 18-month period (April 2018-Septem-

ber 2019). The inclusion criteria of the study con-

sisted of established patients at KNIGHTS Clinic

who signed up for receiving CM texts on their

phones. No other opt-in process was required for

signing up for receiving CM texts. Exclusion crite-

ria consisted of patients who turned down or

opted out of receiving CM texts as well as patients

Figure 1. Sample view of the CareMessage system

Journal of Student-Run Clinics | CareMessage Text Usage Increases Appointment Adherence in a Student-Run Free Clinic

journalsrc.org | J Stud Run Clin 8;1 | 3

whose appointments were confirmed by staff of

GMH instead of KNIGHTS Clinic.

A KNIGHTS Clinic CM account was made on

the CM website. Patients were enrolled into the

system by GMH when they signed up to receive

CM texts. Appointment reminders were set up by

inputting a patient's phone number as well as ap-

pointment date, time, and location. Automated

appointment reminder texts were subsequently

sent to patients two days prior to their appoint-

ments as follows: “Your appointment is on [day

and date] at [time]. Text YES to confirm, NO to de-

cline, or STOP.” Patients whose primary or pre-

ferred language was Spanish received the re-

minders in the Spanish format. A status of “Go-

ing” seen next to the patient’s name on the CM

website indicated that the patient had confirmed

their appointment A status of “Not Going” indi-

cated that they had declined their appointment.

If a patient did not respond to the message, a sta-

tus of “No Response” was shown. A sample view

of the CM system is shown in Figure 1. Phone calls

were made via the Google Voice app (2019-2021,

Google, Mountain View, California), and were only

used for rescheduling, confirming cancellations,

or contacting patients who had not responded to

the CM text.

Patients’ appointment statuses were collected

directly from the KNIGHTS Clinic CM platform.

Appointment attendance was then verified via

the patient electronic medical records (EMR). The

outcome variables of the study for both the

needs assessment (pre-intervention) and post-in-

tervention included the following: 1) average

number of patients scheduled per clinic, 2) aver-

age number of patients seen during each clinic,

3) average number of appointment no-shows per

clinic, 4) average number of confirmed and can-

celed appointments via CM, and 5) average num-

ber of phone calls made to patients weekly. Data

for each variable were collected via reviewing of

the CM website and patient phone logs within

the EMR. They were subsequently entered into

Excel (Version 2203 Build 16.0.15028.20152, Mi-

crosoft, Redmond Washington) spreadsheets

where access was limited to only those who

needed it. No identifying patient information (e.g.

name, demographics) was collected.

The study data did not meet the assumptions

of normality based on the Shapiro-Wilk test;

therefore, a non-parametric test was used for

analysis. The Mann-Whitney U test was used for

comparison of appointment no-shows, number

of phone calls made, and number of patients

scheduled per clinic between the pre- and post-

intervention groups. All data were analyzed at the

conclusion of the data collection using Excel. An

of 0.05 was the threshold used to determine data

significance. A statistician from the University of

Central Florida College of Medicine was con-

sulted for data analysis. The University of Central

Florida Institutional Review Board office ap-

proved this study as not a human study research.

Results

During the needs assessment (pre-interven-

tion), a total number of 104 patients (n=104) were

scheduled for primary or specialty care appoint-

ments, with 89 patients seen at the clinic and 12

no-shows. After the intervention, a total of 243 pa-

tients (n=243) were scheduled, with 209 patients

seen and 8 no-shows. A total number of 137 and

100 phone calls were made to patients pre- and

post-intervention, respectively. The average

number of phone calls made per patient was 1.32

pre-intervention and 0.41 post-intervention, with

a 69% reduction in the number of phone calls

made per patient. The average number of phone

calls made weekly to patients was 9.13 pre-inter

Table 1. Data collected pre- or post-intervention

Variable

Number pre-intervention

Number post-intervention

P-value

Patients scheduled per day

6.93

7.84

0.370

Patients seen per day

5.93

6.74

0.640

No shows per day

0.80

0.26

<0.001

Phone calls made per week

9.130

3.230

0.040

Journal of Student-Run Clinics | CareMessage Text Usage Increases Appointment Adherence in a Student-Run Free Clinic

journalsrc.org | J Stud Run Clin 8;1 | 4

Figure 2. The impact of CareMessage on the

number of patients scheduled and seen and

phone calls made per week

*** = p<0.050

Figure 3. The impact of CareMessage on patient

no-shows per day

*** = p<0.050

vention and 3.23 post-intervention (p<0.001) (Ta-

ble 1, Figure 2). The average number of patients

scheduled per clinic pre- and post-intervention

was 6.93 and 7.84 (p=0.370), respectively (Table 1,

Figure 2). The average number of patients seen

per clinic pre- and post-intervention was 5.93 and

6.74 (p=0.640), respectively (Table 1, Figure 2). The

average number of appointment no-shows pre-

intervention was 0.80 per clinic, while the aver-

age number post-intervention was 0.26 per clinic

(p=0.040) (Table 1, Figure 3). The data suggest a

67.5% decrease in the number of appointment

no-shows after the implementation of the CM

text confirmation system.

Over the 18-month intervention period, a total

of 223 CM text messages was sent to patients,

with 150 confirmations and 25 cancellations via

text. An average of 7.20 text messages were sent

prior to each clinic. The average number of con-

firmed texts was 4.84 per clinic, and the average

number of cancellation texts was 0.81 per clinic.

Discussion

Many studies have assessed the effect of text

message reminder systems on clinic operations

and patient continuity of care in various clinical

settings. Text message reminders compared to

telephone reminders are more cost-effective and

have been shown to improve appointment at-

tendance rates thereby allowing optimal routine

care.

3,6,10

However, few studies have specifically

assessed its application in underserved popula-

tions seen in student-run clinics. To the best of

our knowledge, there have been minimal studies

conducted by free or student-run clinics.

Previ-

ous research conducted with underserved adult

patients in a community health center and low-

income, minority pediatric patients in commu-

nity-based clinics demonstrated significantly in-

creased attendance to their appointments with

text message appointment reminders.

7,9

Our pro-

ject explored the role of the CM text confirmation

system in reducing appointment no-shows

among underserved patients. The study findings

agree with those from other studies evaluating

text message-based reminders in other primary

and specialty care clinics.

1-8

In our student-run

free clinic, the implementation of a text-based re-

minder system effectively reduced the number

of appointment no-shows. In addition, the CM

system resulted in fewer phone calls to patients

Journal of Student-Run Clinics | CareMessage Text Usage Increases Appointment Adherence in a Student-Run Free Clinic

journalsrc.org | J Stud Run Clin 8;1 | 5

and more patients seen at the clinic each week.

These findings suggest the increased efficacy of

a text-based reminder system in comparison to

other methods of communication such as direct

phone calls and voicemails.

The observed increase in appointment compli-

ance is likely related to increased patient conven-

ience in responding to appointment reminders

and text messaging being a preferred method of

communication. Possible challenges with direct

phone calls included conflicts with patient work

schedules and subsequent inability to answer the

phone, as well as discrepancies in the call-back

numbers they received; some patients only had

the direct line to GMH instead of KNIGHTS Clinic’s

own Google Voice number. Although there are

multiple likely explanations, further study is re-

quired to explore reasons why patients prefer

text-based reminders to conventional phone

calls and voicemails. Additionally, a survey on pa-

tient satisfaction with the CM system could pro-

vide valuable information to optimize the plat-

form at KNIGHTS Clinic. Other variables that may

influence the reduction in appointment no-

shows may involve scheduled care types or sea-

sons. For instance, patients participating in fol-

low-up or specialty care may be more likely to

show up to their appointments due to the rela-

tionship they have established with the

healthcare provider and the acuity of their medi-

cal condition, respectively. Patients may also be

more likely to show up to appointments during

the flu season. However, these variables are less

likely confounders as the study data were col-

lected year-round.

The CM system substantially decreased the av-

erage number of phone calls made weekly to pa-

tients which allowed for additional clinic prepara-

tion time and more scheduled patient appoint-

ments per clinic. The number of patients seen at

each clinic did not change significantly; this could

be attributed to attending physician no-shows.

Additional research should be conducted to

identify other contributing factors. The reduction

in appointment no-shows enhanced patient con-

tinuity of care and access to care due to increased

patient attendance at our clinic. Increased pa-

tient continuity of care may also improve patient

care satisfaction; however, additional studies

should be performed to examine the relationship

between continuity of care and patient satisfac-

tion. The significant reduction in no-shows also

provided more learning opportunities for stu-

dents involved in patient care. Although not

quantitatively measured, the CM system may

also contribute to improved clinic workflow, as

student care coordinators were able to assist the

clinic staff with patient check-ins and registra-

tions. Further research on the impact of the CM

system on clinic workflow is necessary to support

this conclusion and to explore the relationship

between text reminders, enhanced patient out-

comes, and student education.

An important limitation of this study is that no

patient demographic data was collected since

this study was originally a quality improvement

project. An additional study incorporating pa-

tient demographics would allow for the assess-

ment of the reproducibility of the results of this

study and external validity. The CM platform does

not indicate whether a patient has read the text

message or not. Therefore, it could be difficult to

determine whether a patient had a recent

change in phone number, leading to loss to fol-

low-up. The CM platform also does not show the

percentage of patients who had opted out of re-

ceiving text messages, and no data was collected

to indicate the percentage of patients who had

opted in to receive text messages. Therefore, fur-

ther evidence for the utility of this technology for

most patients cannot be confirmed, but this can

be addressed in future studies. Another im-

portant limitation of this study is the difference in

sample size between the pre- and post-interven-

tion groups. To address this limitation, a future

study should enroll more patients and run for a

longer duration to obtain more data. Further data

collection and analysis would also increase the ef-

fect size of the study. Finally, although the study

data demonstrated statistical significance in the

reduction of appointment no-shows, the clinical

significance of this result may be questionable as

both pre- and post-intervention groups averaged

<1 patient no-show per clinic day. However, the

study data demonstrated both a statistical and

clinical significance in the reduction of phone

calls made weekly to patients.

The CM text message system is a simple appli-

cation interface that sends scheduled and auto-

mated appointment reminder messages to

Journal of Student-Run Clinics | CareMessage Text Usage Increases Appointment Adherence in a Student-Run Free Clinic 
journalsrc.org | J Stud Run Clin 8;1 | 6 
patients and allows for monitoring of patient re-
sponses.  Clinics  hoping  to  implement  such  a 
platform  should  keep  in  mind  the  following:  1) 
they must register patients for receiving CM texts 
on their phones; 2) they must verify that the pa-
tient’s  number  is  a  mobile  number  and  not  a 
landline; 3) patients may opt out of receiving CM 
texts at any time by replying “STOP” to the text 
message  system;  4)  the  clinic  staff  may  com-
municate  with  patients  directly  in  real-time  to 
help  address  any  concerns;  5)  the  system  only 
provides automated messages in the English or 
Spanish  language,  thus  messages  sent  to  pa-
tients  who  are  non-English  or  non-Spanish 
speaking must be individualized to their respec-
tive languages; and  most importantly, 6)  the el-
derly population may not have sufficient techno-
logical literacy, and proper education, such as via 
handouts, on operating text messaging applica-
tions on their phones must be provided to them 
if  needed.  Although  beyond  the  scope  of  this 
study, the CM system also provides other forms of 
patient  communication,  including  the  ability  to 
do  the  following:  1)  deliver  pre-appointment  in-
structions  such  as  fasting  for  laboratory  blood 
draws;  2)  collect  patient  satisfaction  surveys  re-
garding the use of the CM platform; 3) provide ed-
ucational information to patients such as dietary 
restrictions appropriate for managing their blood 
pressure or smoking cessation; and 4) update pa-
tients on the status of their specialty referrals. The 
CM  system  could  be  used  in  the  future  at 
KNIGHTS Clinic to serve in other forms of patient 
communication,  such  as  prescription  pick-up 
and specialty referral reminders; additional stud-
ies should explore these potential uses. 
     Our quality improvement study demonstrates 
that  implementing  text-based  clinic  appoint-
ment  reminders  reduces  the  amount  of  time 
spent attempting to contact patients directly via 
phone  calls,  thereby  creating  additional  clinic 
preparation  time.  More  importantly,  text  mes-
sage reminders decrease no-shows, thereby en-
hancing overall appointment adherence and pa-
tient access to care. While this finding has been 
studied in several clinical scenarios, our project is 
the first to  explore  the  use of  the CM system in 
the  context  of  a  low-resource  patient  demo-
graphic seen at a student-run clinic. Overall, the 
CM  system  may  positively  impact  patient  care 
satisfaction,  student  learning,  and  clinic  work-
flow; and further studies examining the relation-
ships between the CM system and these factors 
are encouraged. We recommend this communi-
cation platform to other student-run free clinics 
that are looking to increase clinic efficiency and 
improve patient continuity of care. 
 
Acknowledgements 
The authors would like to acknowledge the UCF College of 
Medicine, Diebel Legacy Fund at Central Florida Foundation 
for funding the clinic activity, Grace Medical Home, Marvin 
Hardy M.D., UF College of Pharmacy, and the countless phy-
sician and student volunteers who have made this project 
possible. Also, Dr. Judith Simms-Cendan for her time, men-
torship, and direction as our faculty advisor. 
Disclosures 
The authors have no conflicts of interest to disclose. 
References 
1.  Anthony  N,  Molokwu  J,  Alozie  O,  Magallanes  D.  Imple-
mentation  of  a  text  message  to  improve  adherence  to 
clinic and social service appointments. J Int Assoc Provid 
AIDS Care. 2019 Jan-Dec;18:2325958219870166. LINK 
2.  Guy R, Hocking J, Wand H,  Stott S,  Ali H, Kaldor J. How 
effective are short message service reminders at increas-
ing  clinic  attendance?  A  meta-analysis  and  systematic 
review. Health Serv Res. 2012 Apr;47(2):614-32. LINK 
3.  Leong KC, Chen WS, Leong KW, et al. The use of text mes-
saging to improve attendance in primary care: a random-
ized controlled trial. Fam Pract. 2006 Dec;23(6):699-705. LINK 
4.  Norton BL,  Person AK,  Castillo  C, Pastrana C,  Subrama-
nian M, Stout JE. Barriers to using text message appoint-
ment reminders in an HIV clinic. Telemed J E Health. 2014 
Jan;20(1):86-9. LINK 
5.  Cole-Lewis H, Kershaw T. Text messaging as a tool for be-
havior change in disease prevention and management. 
Epidemiol Rev. 2010;32(1):56-69. LINK 
6.  Junod Perron N, Dao MD, Righini NC, et al. Text-messag-
ing  versus  telephone  reminders  to  reduce  missed  ap-
pointments  in  an  academic  primary  care  clinic:  a  ran-
domized controlled trial. BMC Health Serv Res. 2013 Apr 
4;13:125. LINK 
7.  Percac-Lima S,  Singer  DE,  Cronin  PR,  Chang  Y,  Zai  AH. 
Can text messages improve attendance to primary care 
appointments  in  underserved  populations?  J  Health 
Care Poor Underserved. 2016;27(4):1709-25. LINK 
8.  Starnes JR, Slesur L, Holby N, Rehman S, Miller RF. Pre-
dicting  no-shows  at  a  student-run  comprehensive  pri-
mary care clinic. Fam Med. 2019 Nov;51(10):845-9. LINK 
9.  Stockwell  MS,  Kharbanda  EO,  Martinez  RA,  et  al. 
Text4Health: impact of text message reminder-recalls for 
pediatric  and  adolescent  immunizations.  Am  J  Public 
Health. 2012 Feb;102(2):e15-21. LINK 
10.  Gurol-Urganci I, de Jongh T, Vodopivec-Jamsek V, Atun R, 
Car  J.  Mobile  phone  messaging  reminders  for  attend-
ance  at  healthcare  appointments.  Cochrane  Database 
Syst Rev. 2013 Dec 5;2013(12):CD007458. LINK 