Running head: EVALUATION OF APPOINTMENT REMINDER SYSTEM 1

Running head: EVALUATION OF APPOINTMENT REMINDER SYSTEM

1

Evaluation of a Reminder System for Perinatal Psychiatric Appointments

Lacy R. Clayton, PMHNP-BC, MSN, RN

Doctor of Nursing Practice

DNP Project Chair: Patti Rager Zuzelo

Faculty Advisor: Joanne Serembus

Drexel University

September, 2018

EVALUATION OF APPOINTMENT REMINDER SYSTEM

2

Ó Copyright 2018

Lacy Rae Clayton, All Rights Reserved


3

Table of Contents

Page

Abstract 7

Chapter One: Introduction and Overview of the Problem 8

Problem Statement 10

Current Practices 10

Purpose of the DNP Project 11

Project Objectives and Outcomes 11

Clinical Question (PICOT) 12

Chapter Two: Review of Literature/Synthesis of Literature 13

Search Table 12

Appraisal of Evidence 14

Limitations 21

Summary & Conclusion 22

Chapter Three: Conceptual Foundation or Model 23

Conceptual/Theoretical Model 24

Conclusion 25

Chapter Four: Project Design 25

Project Implementation Steps 25

Practice Setting 26

Key Informants 27

Practice Change/Quality Improvement Design 27

Data Collection Tools 27


4

Data Collection Methods 28

Institutional Review Board (IRB) Approval 28

Conclusion 29

Chapter Five: Implementation Processes/Procedures 29

Letters of Permission 30

Resources 30

Conclusion 30

Chapter Six: Evaluation and Outcomes of Project 31

Key Informants 31

Methods 31

Data Analysis 32

Results 32

Conclusion 35

Chapter Seven: The Essentials of Doctoral Education for Advanced Nursing Practice 36

Implications to Practice/Policy Changes 36

Essential I: Scientific Underpinnings 37

Applicability to Project 37

Implications for Practice 37

Essential II: Organizational and Systems Leadership 37



Essential III: Clinical Scholarship and Analytical Methods 38


5



Essential IV: Information Systems/Technology 39



Essential V: Healthcare Policy for Advocacy 40



Essential VI: Interprofessional Collaboration 41



Essential VII: Clinical Prevention and Population Health 42



Essential VIII: Advanced Nursing Practice 42



Chapter Eight: Project Implications and Conclusions 44

Project Strengths and Limitations 44

Significance/Implications 45

Conclusion 45


6

References 46

Appendices 54


7

Abstract

Background: Rates for missed appointments (no-shows) are especially high in psychiatry.

Perinatal women are particularly at risk for missing appointments due to logistics such as

childcare and transportation. Though rates of depression are higher for perinatal women than the

general population, perinatal women often remain untreated. Appointment reminder systems are

an effective method for increasing appointment attendance and cancellations. Literature suggests

providing two reminders within seven days of the appointment. Maintaining accurate patient

contact information is a challenge to delivering reminders.

Purpose: Compare the appointment reminder system of a large, urban hospital to evidence-based

practice in an effort to reduce the 60% no-show rate within its perinatal psychiatry service.

Methods: A two-hour work-group meeting with five identified key experts evaluated the

appointment reminder system within the perinatal psychiatry service. A semi-structured

interview compared the intended appointment reminder process to its actual delivery. Strengths

and challenges were identified.

Results: The institution’s appointment reminder system consists of one automated reminder 48

hours prior to the appointment. The scheduling database is not updated to reflect patients who

cancel appointments, causing available appoint times to remain unfilled. Depending on the

referral source, some patients receive additional personal reminders from clinicians.

Conclusions: The current appointment reminder system may benefit from adding an additional

reminder based on evidence-based practice which suggests two reminders are preferred.

Attention should be given to appointment cancelations to optimize provider time and increase

patient access.

Keywords: appointment reminders, SMS reminders, appointment no-show.


8

Evaluation of a Reminder System for Perinatal Psychiatric Appointments

Chapter One: Introduction and Overview of the Problem

Inadequate treatment of mental-health diagnoses is prevalent in the United States. Nearly

50% of Americans experience depression, anxiety or another mental health diagnosis throughout

their lifetime (Andrews & Thomson, 2009). Depression is the leading cause of disability for

persons age 15-44 in the United States (National Institute of Mental Health, 2018). The National

Alliance on Mental Illness (NAMI) reports an estimated 60% of Americans with mental illness

remain untreated (2018). Minority groups and underserved populations are especially at risk for

undertreatment. Studies indicate that only 21% of minorities meeting criteria for major

depression receive treatment for their illness (Gonzalez et al., 2010; NAMI, 2018).

Perinatal women are particularly vulnerable as it relates to mental health. The frequency

of depression among postpartum women is higher than depression rates for the general

population, with one in seven women receiving a diagnosis of postpartum depression (American

Psychiatric Association, 2017) compared to one in ten women within the general population

(Centers for Disease Control, 2017). Yet the mental health needs of this group remain

underdiagnosed and undertreated (Geier, Hills, Gonzales, Tum & Finley, 2014).

Inadequate treatment of perinatal mental health has a substantial impact on both mother

and child. Untreated postpartum depression contributes to decreased mother and child bonding

(Moehler, Brunner, Wiebel, Reck & Resch, 2006). Poor maternal bonding significantly impacts

infant development and safety. Mothers with unmanaged mental illness are less likely to engage

in stimulating activities such as reading, playing, and affectionate touching. These mothers are

less attuned to their child’s cues regarding hunger or illness and may not pay proper attention to

safety matters such as child proofing or car seat safety (Field, 2010). Children whose mothers


9

have untreated depression are more likely to be diagnosed with depression and oppositional

defiant disorder during their adolescence (Byatt et al., 2012).

Weinreb et al., (2014) found that 35% of pregnant women with a documented diagnosis

of depression did not receive mental health treatment during their pregnancy. Similarly, Geier et

al. (2014) found only 48% of pregnant women with a diagnosis of depression received treatment

compared to 72% of non-pregnant women diagnosed with depression. The main predictor for

the duration of a postpartum depression episode is the amount of time lapsing before treatment is

initiated, making early detection and treatment crucial for these women and their families

(Hansotte, Payne & Babich, 2017).

Women requiring evaluation and treatment for postpartum depression receive necessary

services only when they present to providers; high no-show appointment rates represent a serious

underutilization of important mental health services with potentially far-reaching adverse effects

on maternal-child health outcomes. Research supports that perinatal women are particularly at

risk for missing scheduled appointments due to logistics such as transportation and childcare

(Grote et al., 2015).

Given the need for mental health treatment, it is important to ensure follow-through with

appointments once they are scheduled. Though missed appointments are experienced across all

medical specialties, evidence suggests that the rate of psychiatric missed appointments may be

twice that of other specialties (Mitchell & Selmes, 2007). Reported rates for psychiatric missed

appointments range from 15%-28% with rates for missing initial appointments typically being

higher than rates for follow-up appointments (Mitchell & Selmes, 2007).

The financial impact of missed appointments is substantial, with an average loss of $200

for each individual missed appointment within a healthcare agency and a total loss of 150 billion


10

dollars annually to the United States healthcare system (Kheirkhah, Feng, Travis, Tavakoli-

Tabasi, & Sharafkhaneh, 2016; Zimmerman, 2014). This is particularly relevant to psychiatry

considering untreated mental health diagnoses are said to cost 100 billion dollars annually in lost

productivity (Scientific American, 2012). Additionally, missed appointments are associated with

poorer health outcomes, including reduced rates of medication adherence (Mitchell & Selmes,

2007), increased symptoms, greater risk of hospitalization with longer stays and increased

hospital readmissions rates (Gajwani, 2014). Rates of missing initial appointments are higher

than non-attendance rates for follow-up appointments (Mitchell & Selmes, 2007).

Problem Statement

Missed appointments have a substantial impact on the financial well-being of the

healthcare system and on the physical and emotional well-being of patients not receiving their

scheduled care. Appointment absenteeism is of particular concern in a psychiatric setting given

this specialty’s high missed appointment rate and the impact of untreated mental health on

patients and society. Although evidence exists on best practices for appointment reminder

systems to improve appointment adherence, these recommendations are not universally

implemented across healthcare settings (McLean et al., 2014). The Department of Obstetrics and

Gynecology at a large teaching hospital in Philadelphia, PA currently has a 35% overall no-show

rate for their obstetrics practice. This rate increases to 60% for missed appointments within the

perinatal psychiatric service from January to August, 2018.

Current Practice

Appointment reminder systems are common practice for reducing the amount of missed

appointments. Reminder systems such as mailings, telephone calls, and text messaging all show

some degree of improving appointment adherence (Finkelstein, Liu, Jani, Rosenthal, &


11

Poghosyan, 2013). Research findings support that tailoring the reminder system based on patient

preferences and utilizing multiple modalities results in higher rates of appointment adherence

(Crutchfied & Kistler, 2017).

While developing this DNP project, it was unknown if the institution’s appointment

reminder system adhered to evidence-based best practices or what challenges were experienced

related to the system. This prompted an evaluation of the reminder to gain insight on potential

improvements which could thereby increase appointment attendance and patient access to care.

Purpose of the DNP Project

The purpose of this Doctor of Nursing Practice (DNP) evidence-based project was to

complete an evaluation of the appointment reminder system for perinatal psychiatric

appointments within a large urban hospital’s outpatient obstetrics clinic. The evaluation was

comprised of two steps. First, the design of the current reminder system was evaluated

compared to the current literature regarding best practices for appointment reminder systems.

Second, process mapping based on a key informant work group was completed to provide a

detailed understanding of how the reminder system is actually delivered and what challenges are

experienced. These two steps identified gaps between the current program design and evidence-

based practice and identified gaps between the intended program design and the delivered

program.

Project Objectives and Outcomes

The main objective of this DNP project was the completion of a comprehensive process

evaluation of the appointment reminder system for perinatal psychiatric appointments scheduled

within this obstetrics clinic. This evaluation included acquiring an understanding of how the

reminder system is intended to be implemented and determining how this compares to the


12

delivered reminder system. This information was gathered through a one-time work group

meeting with key informants. The delivered system is represented by a flow chart/process map

diagram constructed in consultation with key informants to provide a detailed, accurate pictorial

representation of the delivered appointment reminder program. The process map was compared

to current evidence-based practices for optimizing appointment adherence. Recommendations

were solicited from key informants. The primary outcome was an Executive Summary of main

findings and recommendations. The Executive Summary was distributed to key administrative

stakeholders within the institution with the purpose of offering evidence-based solutions for

revising the appointment reminder system with the intent of increasing appointment adherence.

Clinical Question/PICOT

This DNP project evaluated the current appointment reminder system of a perinatal

psychiatric service within an outpatient obstetrics clinic located in a large urban hospital in

Philadelphia. Psychiatry is the clinic’s most requested referral and it currently offers co-located

psychiatry appointments by a psychiatric nurse practitioner one day per week. The psychiatry

service has a no-show rate as high as 61% despite the requested demand. This program

evaluation offers a comprehensive understanding of the current appointment reminder system,

comparing it to the existing literature regarding best practices for appointment reminder systems

and soliciting input from key experts. The clinical question to be answered is, how does the

program’s current appointment reminder system compare to evidence-based practices for

optimizing appointment adherence? This question was constructed using the PICOT format:

Population: Perinatal women scheduled for psychiatry appointments; Intervention: Evidence-

based evaluation of current reminder system; Control: Current reminder system; Outcome:

Improved appointment attendance rates; and Timeline: Four weeks.


13

Chapter Two: Review of the Literature

A comprehensive literature review examining best practices for appointment reminders

was conducted. PubMed, CINAHL, and Cochrane Library databases were searched. Key search

words included: appointment reminder systems, text/SMS reminder system, no-show,

appointment adherence, and improving appointment attendance. Specific information regarding

key search terms and number of retrievals by database can be found in Table 1.

Search Results

Table 1A PubMed Search Terms and Search Results Search terms Number of hits Improving appointment attendance 85 Increasing appointment attendance 42 Appointment reminder 373 Text AND appointment reminder 151 SMS AND appointment reminder 70 Mental health AND no-show 29 Search range 2007-2018 Table 1B CINAHL Search Terms and Search Results Search terms Number of hits SMS reminder 25 Text reminder 14 Appointment attendance 58 Appointment adherence 30 Appointment reminder 87 Text AND appointment AND reminder 70 Mental health AND no-show 15 Search range 2007-2018 Table 1C Cochrane Library Search Terms and Search Results Search terms Number of hits


14

SMS reminder 6 Text reminder 15 Appointment reminder 10 Search range: All

Studies researching telephone and/or text appointment reminder systems to promote

appointment attendance were reviewed. A total of 151 abstracts were returned based on key

search terms. These articles were reviewed based on established inclusion and exclusion criteria.

Eligible articles were published within the past 10 years. To maintain cultural and language

similarities, studies were from North America, The United Kingdom, and Australia. Eligible

studies focused specifically on routine appointment reminders. Studies of reminder systems for

promoting medication adherence or prompting patients to adhere to highly regimented

appointment schedules such as HIV treatment or dialysis were excluded. Studies focusing on

rural areas were also excluded considering this project took place in an urban setting. Studies

were critically appraised for quality and research design and preference was given to randomized

controlled trials and experimental designs. After assessing abstracts for duplicates, design

quality, and meeting inclusion criteria, 16 articles were included for review.

Appraisal of Evidence

The effectiveness of appointment reminder systems has been the subject of an extensive

body of research. Based on the established inclusion and exclusion criteria, 16 articles were

identified as the most relevant to support and guide this DNP project. Five of the studies were

systematic reviews or meta-analyses. Four randomized controlled trials (RCT) and one

prospective, randomized, parallel design trial were included. One study was a quasi-

experimental design. The final five studies consisted of Level III or IV observational cross-


15

sectional, case controlled or convenience sample designs. Refer to the Table of Evidence

(Appendix A) for additional information.

It is established that appointment reminders are effective. A synthesis of 31 RCTs and 11

systematic reviews found that reminders consistently improve appointment attendance (McLean

et al., 2014). However, there is no current definitive best practice guideline for appointment

reminders. In their systematic review, Gurol-Urganci et al. (2013) found that telephone and text

message reminders were equally effective in improving appointment adherence. This literature

review identifies best practices and subsequently evaluates how these practices compare to usual

care within the institution.

Systematic reviews. The five systematic reviews had overarching goals as well as their

own unique perspectives. The number of studies included in each review ranged from eight to

sixty. There was duplication of studies included across the systematic reviews. Gurol-Urganci,

de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) only included RCTs in their Cochrane

Review (N=8), while the other four reviews included a combination of RCTs and other study

designs in their analysis. Three of the reviews focused on digital or text message reminders

while two reviews combined telephone and text message reminders. Each of the reviews

concluded that reminder systems were effective tools for improving appointment adherence.

Telephone reminders. Automated or live telephone calls are perhaps the most common

modality for reminder systems and they are consistently found to be effective. The question of

how many and what format is an important consideration for optimizing a reminder system.

Shah et al. (2016) conducted an RCT to study the impact of a personal reminder call made seven

days prior to the patient appointment compared to usual care of an automated reminder three

days prior to the appointment. The intervention arm had a significantly lower no-show rate than


16

usual care, 22.8% compared to 29.2%. The intervention arm was also more likely to cancel

their appointment ahead of time, allowing opportunity for another patient to be scheduled. Teo,

Forsberg, Marsh, Saha and Dobscha (2017) studied a convenience sample of patients (N = 250)

at the Department of Veteran’s Affairs. They found patients receiving live reminder calls were

significantly more likely to attend appointments than patients receiving a reminder message or

no reminder, with no-show rates of 3%, 24% and 39% respectively. Parikh et al., (2010)

performed a prospective, randomized, parallel design clinical trial comparing three appointment

reminder groups – personalized telephone reminder, automated telephone reminder, and no

telephone reminder. The study confirmed prior findings that no-show rates are significantly

higher for initial appointments compared to follow-up appointments. No-show rates were

significantly lower in the personalized (13.6%) and automated (17.3%) intervention compared to

no intervention (23.1%). Cancellation rates were also significantly higher in the two intervention

groups compared to the control group.

These studies support the effectiveness of telephone reminders for increasing

appointment attendance. Though telephone reminders are effective in general, live or

personalized reminders are more effective than automated reminders. The two RCTs had sample

sizes of N=2,000 or greater and used a 95% confidence interval (CI) to determine differences

between the groups. Teo et al., (2017) studied a more homogenous sample within the

Department of Veteran’s affairs, though Chi-Square analysis revealed similar findings regarding

the effectiveness of telephone reminders.

Text reminders. Multiple studies have investigated the effectiveness of text-based

appointment reminders. Boksmati, Butler-Henderson, Anderson, and Sahama (2016) conducted

a meta-analysis of text reminders that included 28 studies. All included studies found that text


17

message reminders improved appointment adherence. Most reminders were sent within 48-hours

of the scheduled appointment and 75% of the settings sent only one reminder. Most (88%) of the

reminders were generic rather than personalized. These researchers did not find any subgroup

differences for timing, number, or type (automated or personal) of reminders. This may be

related to challenges in comparing the different study designs and the small number of studies

that implemented multiple reminders and/or live reminders. A systematic review by Robotham,

et. al., (2016) had similar findings. The authors’ meta-analysis of 26 articles found that patients

receiving text reminders were 23% more likely to attend appointments than patients who

received no reminder.

Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) conducted a

Cochrane Review of eight RCTs. They found moderate evidence that text reminders improved

appointment attendance compared to no reminders. They also found moderate evidence that

telephone and text reminders had similar outcomes. A systematic review of 60 studies analyzed

the use of text messages to improve various health outcomes, including 11 studies investigating

the impact of text messages on appointment attendance (Kannisto, Kolvunen, and Valimaki,

2014). Attendance rates improved in 77% of the studies.

Studies analyzing text reminders were all systematic reviews or meta-analyses of prior

studies. The effectiveness of text reminders for improving appointment attendance was

consistent across studies. It is important to note there is some overlap of studies included within

the reviews. Inclusion criteria and search methods were clearly defined in each study. Authors

consistently used a 95% CI to determine the significance of differences between groups.

Reminder plus. The literature suggests multiple reminders are optimal for improving

adherence with two reminders increasing attendance up to 19% over one reminder. (Robotham et


18

al., 2016). Two or more telephone contacts or combination telephone/text reminders were most

often evaluated in reminder plus studies. Callinan et al. (2017) found that a personal telephone

call one month prior to recommended follow-up along with a personal telephone reminder the

day prior improved appointment adherence over those participants receiving usual care of a

mailed reminder followed by an automated telephone call the day prior. Clouse, Williams, and

Harmon (2015) used convenience sampling to determine that subjects (N = 18) had increased

appointment attendance following two personal telephone contacts compared to no reminder for

the same time period the year prior. The researchers utilized a “telephone engagement protocol”

process map to study the effectiveness of a personalized phone call about one week prior to the

scheduled appointment, followed with an appointment reminder the day prior to the appointment.

Eighty-percent of patients who received both contacts attended their scheduled appointment,

resulting in a 26% reduction in no-shows compared to the same time the prior year in which no

reminders were provided. In some instances, subjects in the intervention only received one of

the reminders due to technical challenges or issues with contact information. Findings revealed

that receiving both reminders was statistically beneficial over receiving only one of the intended

reminders.

Shah et al. (2016) studied the effectiveness of a live reminder call seven days prior to the

scheduled appointment, combined with usual care of an automated telephone call the day prior to

the appointment. The live call encouraged patients to make concrete planning about their

appointment such as considering transportation and what they were scheduled to be doing

immediately before their appointment. The no-show rate was significantly lower in the

intervention group, 22% compared to 29% in the control group.


19

Lin et al. (2016) found that participants receiving text reminders three days prior to

appointments along with the usual care of day prior automated telephone reminder messages had

a 23.5% no-show rate compared to 38.1% no-show rate for participants receiving only the day

prior automated call. Additionally, participants in the intervention group were more likely to

cancel or reschedule their appointments, allowing the opportunity for other patients to be

scheduled into those times.

Percac-Lima et al. (2016) investigated the impact of text reminders sent seven and one

day prior to the scheduled appointment in addition to the usual care of a live telephone reminder

two days prior compared to usual care only. In the intention to treat (ITT), 17% of patients did

not have a mobile number listed in the EMR which perhaps contributed to a lack of significant

findings between the two groups. When the authors completed a matched-control analysis

including only those who agreed to receive the text reminders, there was a 32% reduction in

missed appointments.

Reminder plus was the most frequently studied intervention in this literature review. A

combination of systematic reviews, RCTs, and convenience sampling were reviewed to

determine the effectiveness of these interventions. Sample sizes ranged from N = 17 to N =

8,425. A CI of 95% was the standard for analysis. Chi-square analysis and flow-sheets were

also used for analyzing data. Researchers consistently found that reminder plus significantly

improved appointment attendance compared to single or no reminders. Several of these studies

reported that incorrect patient contact information was a concern and they emphasized the

importance of maintaining up-to-date patient records.

Patient preference. An awareness of patient preference for receiving appointment

reminders helps guide the development of an agency’s reminder system. Telephone and text


20

message reminders have each been found useful with perhaps a modest preference towards

telephone reminders when researchers have directly compared the two (Robotham et al., 2016).

Crutchfield and Kistler (2017), used convenience sampling to complete a survey of adults

(N = 215) through a national survey company. This online survey of appointment reminders

found that participants preferred one email, phone, or text reminder – in that order of preference -

no more than six days prior to the scheduled appointment. Finkelstein, Liu, Jani, Rosenthal,

Poghosyan (2013) surveyed adults (N = 161) using a cross-sectional study design to assess

preferences for five different appointment reminder systems. Telephone and text reminders both

increased appointment adherence though voice messages left on a mobile or home phone number

were significantly preferred over text messages. Patients reporting more familiarity and usage of

text messaging preferred text-based reminders over voice messages.

Tofighi, Grazioli, Bereket, Grossman, Aphinyanaphongs, and Lee (2017) conducted a

six-month prospective cohort study to obtain patients’ opinion of an appointment reminder

system in an office-based Buprenorphine program. Using a convenience sample (N = 93) the

researchers implemented an appointment reminder system that sent text messages seven, four,

and one day prior to the scheduled appointment. Participants in this study preferred text

messaging to phone reminders. Nearly all participants, 97%, stated the reminders helped them to

attend appointments and 100% of participants believed that all patients should receive text

reminders. A key finding from this study was the frequent turnover of mobile phone contact

information, highlighting the importance of regularly updating patient records.

Studies investigating patient preference used observational design and descriptive

statistics to collect data. The findings were mixed, with one study each reporting telephone, text,

or email as the preferred contact method. Each study sample had the potential for selection bias.


21

One sample was of opiate dependent adults, a second sample was primarily low-income women,

and the third sample was highly educated and predominantly Caucasian. None of the studies

applied their findings to determine if incorporating patient preference into the reminder system

would actually increase attendance rates.

Limitations

A primary limitation of this literature review is the rapid advancement of technology

along with ever-increasing patient access to technology. Several of the systematic reviews and

meta-analyses included studies dating back as far as 2005. The changes in technology during

that time might render different results if the studies were conducted today. To maintain current

evidence-based findings, additional studies of appointment reminder systems should be

conducted to determine the results in today’s technological world.

The studies contained in the meta-analyses had varying outcome measures and

interventions making it challenging to identify one best method for an appointment reminder

system. Several of the studies acknowledged that inaccurate or incomplete patient contact

information impacted participants’ ability to receive the intended intervention. Many of the

interventions had no method of tracking if the reminder was actually received. It is unknown

how much these missed interventions impacted results. However, it does provide an opportunity

for clinicians and administrators to be reminded of the importance of maintaining accurate

patient information. A number of studies reported on the usefulness of “reminder-plus” methods

though there is no clear indication of what additional reminder methods are most beneficial.

Though some studies investigated patient preferences for receiving reminders, those

studies did not in turn investigate if appointment adherence improved when people received their

preferred reminder. As a result, clinicians must remain cautious of basing their reminder systems


22

solely on expressed preferences. The majority of the studies investigated appointment adherence

over a limited time-period. One cannot conclude that the observed improvement in adherence

rates will continue over time. It is possible that the increase was related to the novel intervention

with patient response decreasing over time.

Conclusions

Multiple studies support the effectiveness of appointment reminder systems for

improving appointment adherence. The focus of these studies has evolved as more modalities

for reminder systems become available. Early studies investigated the benefit of mailed

reminders whereas in the current research, the focus is primarily on telephone and text-based

reminders with mailings being either obsolete or part of a supplemental reminder system.

While the benefit of text-message reminders is comparable to that of telephone

reminders, there is perhaps a slight patient preference towards live telephone reminders. Though

no exact formula has been identified as the most effective reminder system, a total of two

reminders consisting of either a text and/or telephone contact appears to be the most effective

and preferred modality. Reminders sent between three and seven days and again the day before

the appointment were most often studied. Reminders sent more than seven days prior were not

found to be effective. The studies also found that participant age did not impact their ability or

willingness to receive text-based messages. Participants were not usually concerned about

breaches in confidentiality with text reminders. The vast majority of participants approached for

these studies had access to mobile phones, though maintaining accurate patient contact

information was identified as a challenge.

The reviewed studies represent a diverse sample including urban settings, minority

populations, and a variety of clinic settings including primary care, psychiatry, endocrinology,


23

and ophthalmology. The studies offer consensus that appointment reminders are an effective

method for improving appointment attendance, with text messaging and telephone reminders as

the preferred modalities. In addition to improving attendance rates, reminders also increase

cancellations rates, allowing for waiting patients to be seen in a timelier manner.

Chapter Three: Conceptual Foundation and Theoretical Model

The Institute for Healthcare Improvement (IHI) describes quality improvement (QI) as an

applied science that incorporates expert knowledge with evidence-based improvement methods.

This is a multidisciplinary approach relying on clinical science and systems theory, among other

fields. The first step in QI is to identify the area for improvement followed by identifying how to

determine if a change results in an improvement and outlining what changes can be made to

result in improvement (IHI, 2018). The Center for Public Health Strategy (2018) developed a QI

roadmap which is detailed in figure 1.

This DNP project used process evaluation to addresses the first three sections of this

roadmap. Understanding the current appointment reminder system is the aim. The measure of

improvement is a reduction in no-show rates for scheduled psychiatric appointments within this

population. The ideas for change will be generated based on the information collected by key

informants. Upon completion, the institution will be provided with detailed recommendations on

how to move forward with the test and spread and sustain components.

Aim:What is the

goal

Measure:Does change

lead to improvement

Ideas:What

changes can result in

improvement

Test:PDSA Cycle

Spread and Sustain

Figure 1. Quality Improvement Roadmap, Adapted from Institute for Healthcare Improvement/Open Access


24

Program evaluation is integral to QI. Programs are evaluated at varying stages, from

their development to their ultimate impact. Confirming a program is implemented as intended,

known as a process evaluation, is a crucial QI measure. The Center for Disease Control (2018)

describes a process evaluation as a method for measuring whether a program has been

implemented as prescribed. In his book, Theory Driven Evaluations (1990), Chen defines the

program as intended as normative theory. Causative theory is defined as the way the program is

actually delivered. A process evaluation provides the opportunity to identify discrepancies

between the normative and causative theories within a program and identify opportunities for

improvement. Evaluations address the who, what, when, and where of a program. They provide

information on whether a program is effective and accessible to the target population while also

identifying potential problems with a program’s implementation. A well-executed process

evaluation provides stakeholders with a comprehensive, detailed understanding of a program and

provides information to aid in program improvements (CDC, 2018).

Theoretical Model

Process mapping, also referred to as flow-charting, is a component of process evaluation

and QI. Key informants are asked to provide detailed information about the process of interest.

This mapping uses a visual representation to describe the step-by-step procedures. Process

mapping provides an understanding of how process actually works and gives insight on how the

process can be improved. Additionally, process mapping helps to clarify and define complex

procedures and identify unnecessary, duplicate, or problematic steps. This framework allows

team members to achieve a shared understanding of the current process and work together on

their goals for process improvement (IHI, 2018). The final product is displayed in a Cause and


25

Effect diagram such as an Ishikawa – or fishbone – diagram which provides a concise, visual

representation of findings (Agency for Healthcare Research and Quality, 2018).

Conclusions

Process evaluation is a cornerstone of the QI process, providing a structured approach

for gaining an accurate, comprehensive understanding of an existing program. The evaluator can

determine if a program is being carried out as prescribed and compare the current process to

evidence-based best practices. Incorporating evidence-based practice is a hallmark of nursing

care. The Institute of Medicine’s (IOM) pioneering report, Crossing the Quality Chasm,

identifies evidence-based practice as a core competency of medical practice, with a goal that

90% of clinical decisions be evidenced-based by the year 2020 (IOM, 2001). The benefits of

evidence-based practice include improved patient care, improved health outcomes, and lower

healthcare costs (Melnyk, Fineout-Overholt, Gallagher-Ford, & Kaplan, 2012).

The objective of this process evaluation was to gain a full understanding of the

institution’s appointment reminder system for perinatal psychiatric appointments. Evidence-

based recommendations improving the reminder process were provided to executive staff. The

intended goal is improved access to psychiatric care for perinatal women.

Chapter Four: Project Design

Project Implementation Steps

A well-implemented QI project requires the appropriate selection of the participants,

including all relevant key stakeholders. Staff and clinicians with an active, direct role in the

appointment scheduling and reminder process were identified by the service line director. A

working group of five key informants convened during a two-hour “working lunch” meeting to

discuss the appointment reminder system. A structured meeting agenda identified the


26

information to be gathered to ensure the meeting objectives were achieved. Flip-charts were

used to capture the data. Process maps of the intended system and the delivered system were

constructed. An Ishikawa diagram was used to visually represent the root cause analysis of

factors contributing to the high no-show rate. Findings were compared to current EBP for

optimizing appointment attendance through reminder systems. Recommendations based on

findings were submitted to the institution in an Executive Summary format. The final project

and outcomes were presented to Drexel University project committee members as per

requirements of the university. The finalized paper detailing the project in its entirety was

submitted to the Drexel University Library archives.

Practice Setting

The process evaluation was conducted in an outpatient obstetrics clinic located within a

large, urban, teaching hospital in Philadelphia. The outpatient obstetrics clinic offers co-located

psychiatric services by a psychiatric nurse practitioner one day per week. The women served

within this clinic are predominately minority women under age 25 with lower socioeconomic

status who are Medicaid eligible. Women are referred to psychiatry directly from the obstetrics

clinic, usually from their obstetrics provider or via a Centering Pregnancy/Parenting program

located within the institution. Centering referrals are typically made by social workers or nurses.

Centering Pregnancy is a program in which pregnant women meet as a group to complete their

prenatal medical appointments and also learn about topics such as nutrition, lactation and self-

care (Centering Healthcare Institute, 2018). To be eligible for the Centering program, the

pregnancy must be considered high-risk due to either comorbid medical conditions - including

depression - or other risk factors such as substance use, poor access to prenatal care, or poor

nutrition.


27

Key Informants

Staff and clinicians identified by the service line director as performing duties directly

related to the referring, scheduling, and reminding of perinatal psychiatric appointments were

invited to voluntarily participate. The panel was comprised of a multidisciplinary group

including social work, nursing, psychiatry, administration, and call center staff. A total of five

key-informants attended.

Practice Change/Quality Improvement Design

This process evaluation was developed using the CDC’s Framework for Program

Evaluation (1999). This framework identifies the following six-steps for a thorough process

evaluation: engage stakeholders; describe the program; focus the evaluative design; gather

credible evidence; justify conclusions; and share lessons learned. As part of the evaluation,

participants completed process mapping to compare the intended delivery system to the actual

delivery system. The 30 standards of effective evaluation identified by Joint Committee on

Standards for Educational Evaluation guided the integrity of this project (Yarbrough, Shulha,

Hopson, & Caruthers, 2011). These standards require that utility, feasibility, propriety, accuracy,

and accountability be considered in the development and implementation of the evaluation.

Data Collection Tool

The meeting agenda was developed and a copy was provided to all participating

volunteers. A semi-structured interview protocol with probes was used to ensure the objectives

were met (Appendix B). To fully understand the appointment reminder process and identify

possible variations in the process, the appointment referral and scheduling process was

incorporated into the interview protocol for discussion.


28

Data Collection Methods & Evaluation

A group of five key informants were invited to participate in this two-hour work group

meeting. Food was provided. The meeting followed the agenda using the semi-structured

interview protocol to develop a process map providing a clear understanding of the intended

appointment reminder process compared to the delivered reminder process. Key questions

included: what is the reminder type (text, phone, mail); is the reminder automated or personal;

number of reminders; when is reminder sent; what is frequency/method for updating patient

records; is contact/non-contact tracked?

Flip-charts were used to assemble the data and construct a process map based on

responses. The final agenda item allowed opportunity to confirm that the process map accurately

reflected the key informant’s descriptions and input. Using the process map, current practice

was compared to the known evidence-based practices for optimizing appointment attendance.

An Ishikawa diagram was constructed to visually present the root cause analysis. An Executive

Summary including recommendations for implementing and evaluating the suggested changes to

the current reminder system process was presented to the program’s service line director.

Institutional Review Board Preparation

The necessary IRB approvals were obtained from the participating institution and Drexel

University. This project was considered exempt and was approved based on a Letter of

Determination between the institution and Drexel University. Key informants were purposively

selected via the obstetrics department. All participation was voluntary. No identifying

information or patient records were accessed. The risks involved in this DNP project were

minimal.


29

Conclusion

This DNP QI project was based on established process evaluation criteria, incorporating

process mapping to provide a rich understanding of the current appointment reminder system.

Purposively selected key informants participated in a work group meeting to develop a thorough

understanding of the reminder system for perinatal psychiatric appointments within this urban

obstetrics clinic.

Chapter Five: Implementation Process and Procedures

This evidence-based QI project was designed to optimize the current appointment

reminder system within a perinatal psychiatric service. The primary implementation was a two-

hour work group meeting with five key informants with direct knowledge of the appointment

scheduling and reminding system. The intended outcome was an increase in appointment

adherence and patient access to care.

The implementation stage of this QI project involved the collaboration of a

multidisciplinary team within the institution. Medical providers from nursing and medicine,

social workers, administrators, technology support, and administrative staff each had integral

roles. Participants from social work and nursing identified the high no-show rate as a primary

concern during an initial information gathering meeting. This information was used to aid in the

development of the resulting evaluation of the appointment reminder system.

Administration identified the appropriate participants, coordinated scheduling, and

secured the meeting location for the work group. An invitational email describing the purpose of

the work group meeting was developed along with a meeting agenda. A semi-structured

interview protocol was constructed based on the CDC’s Framework for Process Evaluation

(1999).


30

Letters of Permission

A letter of permission authorizing this DNP QI project was provided by the institution’s

Women and Children’s Service Line Director. No other permission letters were required for the

implementation of this project. The DNP project was approved as exempt based on a Letter of

Determination between the two institutions.

Resources

This project required minimal use of resources. The primary required resource was

access to the institution’s internal calendar system to aid in scheduling the work group and

securing meeting space. Administrative staff within the institution agreed to facilitate scheduling

the work group and requesting the necessary meeting space. The practice manager and lead

statistician provided information regarding no-show rates and general demographic data for the

practice.

Conclusion

The development of this DNP project occurred over the course of a 12-month period.

Once program staff identified high no-show rates as a primary concern within the perinatal

psychiatric service, efforts began to construct a relevant DNP project which resulted in this

evidence-based evaluation of the appointment reminder system. The desired outcome was to

increase appointment attendance within the perinatal psychiatric service. The required IRB

protocols were followed to ensure the protection of participants. The participating institution

was provided with an Executive Summary of findings along with detailed suggestions on how to

implement and evaluate the proposed program changes.


31

Chapter Six: Evaluation and Project Outcomes

Key Informants

The department administrator identified staff and clinicians most closely associated with

appointment scheduling and reminding. Five key informants were selected. Invited participants

included: the call center manager responsible for the agency-wide appointment reminder system;

the practice manager responsible for scheduling initial appointments within the obstetrics clinic

and for monitoring appointment cancelations and no-shows; a registered nurse responsible for

referring and scheduling within the obstetrics clinics; a social worker responsible for referring

and scheduling within the centering pregnancy/parenting program; and the psychiatric nurse

practitioner serving as the primary psychiatric provider for this population of women and also

manages the scheduling of all follow-up psychiatric appointments. All five invited participants

attended the two-hour meeting.

Methods

Upon identification, the five selected key informants were invited to attend the work

group meeting via the agency’s internal meeting scheduling program. A detailed email

describing the purpose of the meeting was sent to each invitee. The meeting was held in a

conference room within the institution. Lunch was provided. Participants received a $10 café

gift card as a thank you for their time. Participants introduced themselves and described their

various roles. The semi structured interview protocol was followed and key data were captured

on flip charts. The purpose of the interview was to: understand the intended appointment

reminder process; understand the actual process; probe regarding challenges identified in the

literature such as the process for updating patient contact information or if the system confirms if

reminders are received; and identify challenges and suggestions for improvement. Upon


32

conclusion of the meeting, the data were reviewed with the key informants to ensure

understanding and accuracy.

Data Analysis

Data were organized and analyzed based on the structure of the interview: outcomes

related to intended process; outcomes related to actual process; outcomes related to identified

challenges. The data were summarized to offer a comprehensive understanding of the

appointment reminder system as prescribed and as delivered. The reminder system was

compared to evidence-based practices found in the comprehensive literature review. Relevant

findings were highlighted. No-show rates for patients scheduled via the OBGYN clinic and

Centering Pregnancy/Parenting were provided by the lead statistician.

Results

This perinatal psychiatric service has a no-show rate of 58% for patients scheduled

directly from OBGYN and 61% for patients scheduled from Centering Parenting/Pregnancy,

averaging to a 60% no-show rate. The overall OBGYN clinic no-show rate is 35%. The

completion of this process evaluation provided a comprehensive understand of the appointment

reminder process for the psychiatric service with the goal of reducing this particularly high no-

show rate.

Televox is the software use for the appointment reminder system. All patients receive

one automated reminder 48 hours prior to their scheduled appointment. Patients may choose for

reminders to be via telephone or text. Text is the method most often preferred by patients.

Televox reminders allow patients to confirm or cancel their appointments. Televox then

provides a daily report to schedulers and managers which notes if patients confirm or cancel

appointments. If a patient selects “cancel”, a scheduler must review this and manually remove


33

the appointment from the electronic medical record (EMR) as there is no direct link between

Televox and the institution’s scheduling software.

Televox does not have a process for routinely updating patient contact information. The

system does not confirm receipt of the reminder. The only error message Televox provides is if

a text message is sent to a landline. This will be reflected in the above-mentioned report,

becoming the scheduler’s responsibility to reconcile the contact information.

Perinatal psychiatric appointments are scheduled in one of three ways, see figure 2.

Initial appointments scheduled within the outpatient OBGYN clinic follow the above reminder

system protocol with no noted deviation. The schedulers reported that the initial scheduling

process includes confirming patient contact information and asking them how they prefer to be

reminded about appointments – text or telephone. Beyond this, there is no process for routinely

updating patient information. One scheduler noted that patients are often asked “is everything

the same” rather than having the scheduler provide the information on file to confirm accuracy.

Initial appointments scheduled within the Centering Pregnancy/Parenting group follow

the above reminder system protocol though there are also several informal additions to the

reminder process. The social worker scheduling the appointment sends a personal reminder text

to patients the day prior to the appointment. The social worker also follows up with patients if

they miss their appointment and will aid them in rescheduling. Considering this is not a

standardized component of the reminder system, it is important to note that this additional

contact cannot be guaranteed or confirmed for each appointment. There is no formal system for

updating patient contact information though the frequent contact between patients and providers

may offer more opportunity for updating contact information. The psychiatric nurse practitioner

manages the scheduling of all follow-up appointments. In this scenario, the Televox protocol is


34

followed with no noted deviations. The psychiatric nurse practitioner does not typically confirm

contact information.

A crucial finding was the realization that the schedule is not being updated to reflect

when patients select the “cancel” option via the Televox reminder. This information is available

in the daily Televox report but there is no assigned staff to review the report and update the

schedule accordingly. This results in appointment openings going unfilled rather than being

offered to patients in need. This also results in patients being erroneously categorized as no-

shows.

Initial Appointment Scheduled via Outpatient

OB Clinic Scheduled By Practice

Manager or RN

Contact Information Confirmed at

Scheduling of Initial Appointment

One Automated Televox Reminder 48

Hours Before Appointment

Initial Appointment Scheduled via Centering

ProgramScheduled by Social

Worker

No Formal Confirmation of

Contact Information Though Frequent

Contact With Patient


Hours Before Appointment.

Informal Reminder via Social Worker

Follow-up Appointments Scheduled by PMHNP No Confirmation of Contact information


Hours Before Appointment

Figure 2. Scheduling Process for Perinatal Psychiatric Appointments


35

The psychiatric service within this OBGYN clinic has a no-show rate ranging from 58%-

61% depending on the referral source. The total number of no-shows from January until August,

2018 is 274. While many factors contribute to missed appointments, it has been shown that

appointment reminders improve attendance and also increase cancelation rates allowing for

waiting patients to be seen. Improved attendance also has a financial benefit. The institution has

no current data on the financial impact of these missed appointments. Using the formula of a

$200 loss per no-show, it can be estimated that no-show rates within this perinatal psychiatric

service cost an estimated $80,000 annually.

Conclusions

The appointment reminder system appears to be implemented as intended, with one

automated reminder 48 hours prior to the patient’s appointment. No significant barriers or

challenges related to the use of the system were identified. Several opportunities for

improvement were identified and presented to the institution in an Executive Summary Format

60% Average MissedAppointment Rate for Perinatal Psychiatry Appointments

Incomplete Appointment Information

One Automated Reminder

Not checking cancelled

report

System Error

Unknown If Reminder Received

Inaccurate Patient

Information

Reminder does not provide building/suite information

Occasions of reminder havinginaccurate

appointment information (rare)

No structured process forupdating patient information

Televox is unable to confirm receipt

Research suggeststwo reminders

Preference for personalreminders

No staff assigned to check report

Available appointments go unfilled

Inaccurate inflation ofno-show rate

Figure 3. Ishikawa Fishbone Diagram of Root Cause Analysis


36

(Appendix C). The reminder protocol differs from evidence-based recommendations which

suggests two reminders are preferable. The institution may benefit from adding a second

reminder within seven days of the appointment. Evidence also indicates a preference towards

personalized reminders rather than automated. Given the exceptionally high no-show rate of this

service, having one reminder be a personalized phone call might prove advantageous. It must be

noted than no difference was found in the no-show rate for patients referred via the Centering

Pregnancy/Parenting program despite the report of additional reminders in this program. It is

difficult to determine the relevance of this considering those additional contacts are not

formalized and those patients are typically higher-risk.

Inaccurate patient contact information is a main barrier for optimizing appointment

reminder systems. Implementing a standard procedure for updating patient contact information

increases the likelihood that reminders are received. The most notable finding is the failure to

update the schedule to reflect cancelations. This is an inefficient use of provider time. Patients in

need are left waiting unnecessarily while the no-show rate of the perinatal psychiatric service is

inaccurately inflated. Assigning staff to review the daily Televox report, fill openings, and

accurately reflect if an appointment has been cancelled is crucial to improving the attendance

rate.

Chapter Seven: Implications to Nursing Practice

The foundation of this DNP project is based on the Essentials of Doctoral Education for

Advanced Practice Nurses. Also known as the DNP essentials, these criteria were developed by

the American Association of Colleges of Nursing (AACN, 2006) to provide a clear definition of

the core competencies required for advanced practice nursing. Each of the eight essentials


37

played a crucial role in the development of this DNP project. Upon degree completion, the

essentials continue to guide the DNP trained nurse in their daily practice.

Essential I: Scientific Underpinnings for Practice. Determining a singular definition

for nursing is challenging. Most definitions agree that nursing takes on numerous roles. This is

highlighted in the American Nurses Association (ANA, 2017, para. 1) definition which states,

“Nursing is the protection, promotion, and optimization of health and abilities, prevention of

illness and injury, facilitation of healing, alleviation of suffering through the diagnosis and

treatment of human response. To successfully complete these complex tasks, DNP trained nurses

must become experts in accessing, understanding, evaluating, and implementing the relevant

science and theories in their practice area of expertise.

Applicability to Project. This DNP project was developed using rigorous scientific

principles. After identifying the problem of interest, a thorough review of the literature was

completed to identify best practices for increasing appointment attendance. Research was

critically assessed for quality and relevance using proven critical assessment tools. Trained

librarians assisted in identifying appropriate search terms and keywords to ensure an expansive

literature review. Articles were synthesized to determine best practices.

Implications for practice. Quality nursing practice relies on evidence-based

interventions to reduce practice gaps. The DNP educated nurse is able to identify opportunities

for practice improvement and engage in sound scientific methods to implement and evaluate

quality improvement initiatives. The ability to navigate and critically assess existing literature is

an expectation.

Essential II: Organizational and Systems Leadership for Quality Improvement and

Systems Thinking. Advanced practice nurses often perform their roles in large organizational


38

settings. The structure of the United States healthcare delivery system has resulted in the call for

nursing education that focuses on meeting the needs of the current system, while also ensuring

safe, ethical, and quality nursing practice (Zaccagnini & White, 2017). The DNP educated nurse

is expected to practice to their full nursing capability while maintaining an awareness of the

work-flow and fiscal aspects of the organization and considering how these areas can be

improved.

Applicability to Project. This QI initiative was initiated within a large organization. A

primary task for successful project implementation was the identification of relevant

stakeholders – key informants – to provide information regarding the appointment reminder

system. Meeting with key stakeholders provided the opportunity to identify systemic challenges

faced by the organization in promoting patient adherence to psychiatric appointments. The

assessment of identified system provided the opportunity to disseminate key findings.

Recommendations for implementing and evaluating a rapid cycle improvement project were

provided to the institution with the purpose of improving appointment attendance and patient

access to care.

Implications for practice. The DNP educated nurse is capable of navigating complex

organizations and identifying the relevant stakeholders for desired QI initiatives. They should

feel competent in evaluating a practice area and developing a rapid cycle improvement project

based on an identified practice gap. The DNP trained nurse is able to present findings to the

stakeholders in a clear and evidence-based manner.

Essential III: Clinical Scholarship and Analytical Methods for Evidence-Based Practice.

The DNP educated nurse develops and carries out sound research methodologies to address

identified practice gaps. The DNP nurse uses this knowledge to successfully translate evidence-


39

based findings into practice, evaluating the results along the way. Translating research into

practice is a cornerstone of this DNP essential.

Applicability to Project. Upon identifying the high no-show rate as a practice area of

concern, a PICOT question was constructed to aid in guiding the literature review. The PICOT

format allows nurses to systematically formulate a clinical question and to gather the best

evidence to address the question (Echeverria & Walker, 2014). The literature was synthesized to

focus on the appointment reminder system, directing this project towards a specific area of

evaluation. The evaluation was designed using a known scientific framework for QI

implementation (CDC, 1999). A primary outcome of the evaluation was the dissemination of

results to key stakeholders. This process promoted clinical scholarship and identified evidence-

based practice level changes to reduce no-show rates and increase patient access to care.

Implications for practice. The scientific application of a PICOT questions allows the

DNP trained nurse to critically appraise the literature related to an identified practice gap. The

nurse successfully constructs, implements, and evaluates an evidence-based quality improvement

intervention. Disseminating findings to relevant stakeholders facilitates the translation of

evidence into practice.

Essential IV: Information Systems/Technology and Patient Care Technology for the

Improvement and Transformation of Health Care. The AACN states that, “DNP graduates

are distinguished by their abilities to use information systems/technology to support and improve

patient care and healthcare systems…” (AACN, 2016, p. 12), emphasizing the importance of

technology in providing safe, quality patient care. This essential is especially relevant to this

DNP project considering the automated appointment reminder system, Televox, was the primary

focus of the evaluation.


40

Applicability to Project. A comprehensive literature review of relevant electronic

databases as well as use of the World Wide Web for ongoing online inquiry to identify additional

scientific based resources and gray literature set the foundation for this DNP project. An

understanding of the Televox technology was necessary to construct an interview protocol with

relevant questions with adequate probes. The outcomes of this evaluation include

recommendations for optimizing this technology to improve patient access to care.

Implications for practice. The DNP trained nurse must be competent in the utilization

of technology to promote patient care. This includes electronic medical records (EMR), patient

portals, and other modes of technology intended to advance patient care. These systems often

house data that offers insight into the current state of practice. Technological systems and EMRs

are primary tools used for identifying potential practice areas in need of improvement. The DNP

nurse must be competent in extracting and analyzing this data to promote QI initiatives.

Essential V: Healthcare Policy for Advocacy in Health Care. The American Nurses

Association (ANA) considers patient advocacy as a “pillar” of nursing (ANA, 2018, para. 1).

Healthcare policy has the power to promote or impede the healthcare services provided to

patients. The DNP trained nurse can expect to be at the forefront of patient advocacy (AACN,

2006).

Applicability to Project. Patient advocacy was a consideration throughout this DNP

project, beginning with careful attention to the IRB protocol of both participating institutions.

This project was guided by the intent to improve access to psychiatric care for

underserved/perinatal women. A primary outcome of this project involves advocating for

institutional changes in the current reminder system to increase patient access to care.


41

Implications for practice. Maintaining an awareness of deficits within the healthcare

system is an important component of the DNP role. Through proactive participation in policy

development the DNP trained nurse contributes to the reduction of health disparities and

improved healthcare access. Opportunities for advocacy range from the institutional to the

international level (AACN, 2006).

Essential VI: Interprofessional Collaboration for Improving Patient and Population

Health Outcomes. The DNP trained nurse assumes a leadership role that facilitates

interprofessional collaboration. Healthcare organizations are challenged with the need to

improve the quality of care by instituting standards of care and developing health policy while

also maintaining fiscal responsibility. The DNP trained nurse is qualified to aid organizations in

achieving these goals (AACN, 2006). Considering that multiple disciplines are involved with

patient care, developing effective interprofessional collaboration is necessary to promote

efficient, quality care (Poston, Haney, Kott & Rutledge, 2017).

Applicability to Project. This DNP project brought together participants from pediatrics,

obstetrics, psychiatry, social work, nursing, and information systems. Program administrators

participated in the project development and implementation. This DNP project provided the

opportunity for communication about the appointment reminder process across disciplines.

Considerations specific to psychiatry were highlighted and addressed. The collaboration of

psychiatry with obstetrics is an important aspect of this DNP project. An initiative to improve

the show-rate for psychiatry results in more efficient use of the psychiatric provider’s time and

increases the likelihood that this important collaboration between behavioral health and women’s

health continues.


42

Implications for practice. Research indicates that embedded, collaborative care promotes

increased access to care and improved health outcomes (Van Eck, et al., 2018). Promoting

collaborations through QI initiatives facilitates an interdisciplinary approach to patient care. An

understanding of the varied roles and work flow dynamics across disciplines allows the DNP

trained nurse to identify practice gaps and implement work-flow improvements. Effective

communication and leadership skills are necessary to engage the interprofessional teams

(AACN, 2006).

Essential VII: Clinical Prevention and Population Health for Improving the

Nation’s Health. Health promotion and disease prevention are at the heart of nursing. Nursing

practice is directed by an awareness of population health and social determinants of health.

Engaging with patients with cultural and emotional sensitivity fosters relationships and

positively impacts care (AACN, 2006). Healthy People 2020 identifies health equity, reduced

disparities, health promoting environments, and the promotion of health behaviors as their

fundamental goals (Office of Disease Prevention and Health Promotion, 2018).

Applicability to Project. Untreated mental health has a significant impact on the person

and their community. Social determinants of health such as housing, community support, and

employment opportunities are known to increase the risk for depression. The overarching goal

of this DNP project is to increase access to mental health care for an underserved population.

The evaluation of this specific care delivery system – the appointment reminder system –

highlighted issues such as patient literacy, access to telephones, and knowledge of technology.

Implications for practice. Quality nursing care engages in both treatment and

prevention. It is necessary to treat the entire patient and consider their environmental and social


43

risk factors. This awareness allows the nurse to develop a treatment plan that includes patient

education and prevention strategies for health promotion.

Essential VIII: Advanced Nursing Practice. Advanced nursing practice encompasses each

of the DNP essentials. The advanced practice nurse is expected to have highly developed

assessment skills and sound knowledge of pathophysiology and pharmacology. The APN is

expected to provide quality, evidenced-based, patient-centered care.

Applicability to Project. Advanced nursing practice was present throughout this DNP

project. The PICOT question and literature review were developed using principles of nursing

science. Relationships were established to support the development and implementation of this

QI initiative. The successful completion of this DNP project relied on the engagement of

interdisciplinary participants. Th received evidence-based recommendations to aid in improving

mental health access for postpartum women. This project provides an example of nursing

identifying a practice gap and utilizing advance practice nursing skills to translate evidence into

practice.

Implications for practice. DNP educated nurses are experts in the multiple facets of nursing

practice. They possess the skills to formulate science-based practice questions. The DNP nurse

designs, implements, and evaluates interventions to address these practice questions. Effective

leadership and evidence-based, patient-centered care incorporates the tenets of advanced practice

nursing. This allows for ongoing positive contributions to the ever-growing body of nursing

science. Modeling high-quality nursing practice has a positive influence on nursing and the

multiple other disciplines working closely with nursing.


44

Chapter Eight: Project Implications and Conclusions

This DNP project highlights potential deficits of a large organization’s appointment

reminder system. Several suggestions for improving the current reminder system were provided

to the institution based on the outcomes of this evaluation. Implementing the recommendations

may increase patient access to care and improve the fiscal outcomes of the obstetrics department.

Strengths

An interdisciplinary approach was used to identify the practice concern of no-show rates

within the perinatal psychiatry service. This project is important for nursing practice considering

it involves patient access to care and advocating for patients who are often underserved.

Principles of nursing science were applied to develop the PICOT question and guide the

literature review. The process evaluation was developed using evidence-based techniques for QI

initiatives. The specific focus on the appointment reminder system allowed for a highly specific

evaluation resulting in targeted suggestions for improvement. The finding that cancelled

appointments are not currently noted in reminder provides a notable opportunity for change

Limitations

The appointment reminder system is only one possible contributor to high appointment

no-show rates. Though implementation and evaluation of the recommendations was beyond the

scope of this DNP project, it will be important for future work to evaluate if the suggested

changes actually improve appointment attendance. The institution is encouraged to implement

the rapid cycle improvement strategy as suggested in the Executive Summary. There is no

specific information available on the financial impact of these missed appointments. This

evaluation provides an estimate based on using a monetary amount cited in the literature. With a

no-show rate as high as 60%, awareness of the fiscal impact is important.


45

Significance and Implications

Untreated perinatal mental health is a known practice gap that impacts the mother, the

child, and the community. The current no-show rate for the perinatal psychiatric service puts the

collaboration between psychiatry and obstetrics at risk. Considering this is the most requested

referral within the obstetrics clinic, the need and desire for this service is evident. Key findings

from this process evaluation have the potential to increase appointment attendance and ensure

this important collaboration continues. The findings from this process evaluation may

potentially benefit the institution’s entire Televox system considering the current appointment

reminder system is standardized across all hospital departments.

Conclusions

This DNP project addressed an important practice gap within a large hospital setting.

The project was conducted using comprehensive research methods and nursing science. The

DNP essentials guided the project’s development and implementation. This project provides an

example of translating research into practice with the potential to increase patient access to care,

optimize provider resources, and improve financial outcomes.


46

References

Andrews, P., Thomson, A. (2009). Depression’s evolutionary roots. Scientific American.

Retrieved from: https://www.scientificamerican.com/article/depressions-evolutionary/

Agency for Healthcare Research and Quality (2018). Plan Do Study Act Cycle. Retrieved from:

https://innovations.ahrq.gov/qualitytools/plan-do-study-act-pdsa-cycle

American Association of Colleges of Nursing (2006). The Essentials of Doctoral of Education

for Advanced Nursing Practice. Retrieved from:

http://www.aacnnursing.org/Portals/42/Publications/DNPEssentials.pdf

American Nurses Association (2018). Advocacy. Retrieved from:

https://www.nursingworld.org/practice-policy/advocacy/

American Nurses Association (2018). Scope of Practice. Retrieve from:

https://www.nursingworld.org/practice-policy/scope-of-practice/

American Psychological Association (2017). What is postpartum depression and anxiety?

Retrieved from: http://www.apa.org/pi/women/resources/reports/postpartum-

depression.aspx

American Psychiatric Association, Center for Workplace Mental-Health (2017). Quantifying the

cost of depression. Retrieved from: http://workplacementalhealth.org/Mental-Health-

Topics/Depression/Quantifying-the-Cost-of-Depression

Boksmati, N., Butler-Henderson, K., Anderson, K., Sahama, T. (2016). The effectiveness of

SMS reminders on appointment attendance: A meta-analysis. Journal of Medical Systems,

doi: 10.1007/s10916-016-0452-2

Byatt, N., Simas, T., Lundquist, R., Johnson, J., Ziedonis, D. (2012). Strategies for

improving perinatal depression treatment in North American outpatient obstetric


47

settings. Journal of Psychosomatic Obstetrics and Gynecology, 33(4), 143-161.

Byatt, N., Biebel, K., Friedman, L., Debordes-Jackson, G., Ziedonis, D., & Pbert, L. (2013).

Patient’s views on depression care in obstetric settings: How do they compare to the views

of perinatal health care professionals? General Hospital Psychiatry, 35, 598-604.

Byatt, N., Levin, L., Ziedonis, D., Moore-Simas, Allison, J. (2015). Enhancing participation in

depression care in outpatient perinatal care settings. Obstetrics and Gynecology, 126(5),

1048-1058.

Callinan, C., Kenney, B., Hark, L., Murchison, A., Dai, Y., Leiby, B., Mayro, E., Bilson, J.,

Haller, J. (2015). Improving follow-up adherence in a primary eye care setting. American

Journal of Medical Quality, 32(1), 73-79.

Center for Disease Control (1999). Framework for Program Evaluation in Public Health.

Retrieved from: https://www.cdc.gov/mmwr/PDF/rr/rr4811.pdf

Center for Disease Control (2017). Depression among women. Retrieved from:

https://www.cdc.gov/reproductivehealth/depression/index.htm

Center for Public Health Quality (2018). Step by Step Guide to Implement Quality Improvement.

Retrieved from: http://www.amchp.org/Transformation-

Station/Documents/QI%20Step%20by%20Step%20Guide.pdf

Centering Healthcare Institute (2018). Retrieved from: https://www.centeringhealthcare.org

Chen, H. (1990). Theory-Driven Evaluations. Newbury Park, CA: Sage Publications.

Clouse, K., Williams, K., Harmon, J. (2017). Improving the no-show rate of new patients in

outpatient psychiatric practice: An advance practice nurse-initiated telephone engagement

protocol quality improvement project. Perspectives in Psychiatric Care, 53, 127–134.

Crutchfield, T. M., & Kistler, C. E. (2017). Getting patients in the door: medical appointment


48

reminder preferences. Patient preference and adherence, 11, 141.

Echevarria, I. M., & Walker, S. (2014). To make your case, start with a PICOT question.

Nursing2018, 44(2), 18-19.

Field, T. (2010). Postpartum depression effects on early interactions, parenting and

safety practices: A review. Infant Behavior and Development, 33(1), 1-6.

Finkelstein, S. R., Liu, N., Jani, B., Rosenthal, D., & Poghosyan, L. (2013). Appointment

reminder systems and patient preferences: Patient technology usage and familiarity with

other service providers as predictive variables. Health informatics journal, 19(2), 79-90.

Gajwani, P. (2014). Can what we learned about reducing no-shows in our clinic work for you?,

Current Psychiatry, 13(9), 13.

Geier, M., Hills, N., Gonzales, M., Tum, K., Finley, P. (2014). Detection and treatment

rates for perinatal depression in a state Medicaid population. CNS Spectrums, 20, 11-

19.

González, H. M., Vega, W. A., Williams, D. R., Tarraf, W., West, B. T., & Neighbors, H. W.

(2010). Depression Care in the United States: Too Little for Too Few. Archives of General

Psychiatry, 67(1), 37–46. http://doi.org/10.1001/archgenpsychiatry.2009.168

Grote, N., Katon, W., Russo, J., Lohr, M.J., Curran, M., Galvin, E., Carson, K. (2015).

Collaborative care for perinatal depression in socioeconomically disadvantaged women: A

randomized controlled trial. Depression and Anxiety, 32, 821-834.

Gurol-Urganci, I., de Jongh, T., Vodopivec-Jamsek, V., Atun, R., & Car, J. (2013). Mobile

phone messaging reminders for attendance at healthcare appointments. Cochrane Database

Syst Rev, 12(12), CD007458.


49

Hansotte, E., Payne, S., Babich, S. (2017). Positive postpartum depression screening practices

and subsequent mental health treatment for low-income women in Western countries: A

systematic literature review. Public Health Reviews, 38(3), doi: 10.1186/s40985-017-0050-y

Institute for Healthcare Improvement (2018). Science of Improvement: How to Improve.

Retrieved from:

http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementHowtoImprove.aspx

Institute of Medicine (2001). Crossing the quality chasm: A new health system for the 21st

century. Retrieved from:

http://www.nationalacademies.org/hmd/~/media/Files/Report%20Files/2001/Crossing-the-

Quality-Chasm/Quality%20Chasm%202001%20%20report%20brief.pdf

Kannisto, K., Koivunen, M., Valimaki, M. (2014). Use of mobile phone text message reminders

in health care services: A narrative literature review. Journal of Medical Internet Research,

16(10), doi: 10.2196/jmir.3442

Kaplan-Lewis, E., Percac-Lima, S. (2013). No-show to primary care appointments: Why patients

do not come. Journal of Primary Care and Community Health, 4(4), 251-255.

Kheirkhah, P., Feng, Q., Travis, L. M., Tavakoli-Tabasi, S., & Sharafkhaneh, A. (2015).

Prevalence, predictors and economic consequences of no-shows. BMC health services

research, 16(1), 13. Retrieved from:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4714455/pdf/12913_2015_Article_1243.pdf

Ko, J., Farr, S., Dietz, P., Robbins, C. (2012). Depression and treatment among U.S. pregnant

and nonpregnant women of reproductive age, 2005-2009. Journal of Women’s Health, 21(8),

doi: 10.1089/jwh.2011.3466


50

Kopelman, R., Moel, J., Mertens, C., Stuart, S., Arndt, S., O’Hara, M. (2008). Barriers to

care for antenatal depression. Psychiatric Services, 59(4), 429-432.

Lin, C., Mistry, N., Bonch, J., Li, H., Lazebnik, R. (2016). Text message reminders increase

appointment adherence in a pediatric clinic: A randomized controlled trial. International

Journal of Pediatrics, doi: 10.1155/2016/8487378

McLean, S., Gee, M., Booth, A., Salway, S., Nancarrow, S., Cobb, M., & Bhanbhro, S. (2014).

Targeting the Use of Reminders and Notifications for Uptake by Populations (TURNUP): a

systematic review and evidence synthesis. Health Services and Delivery Research, 2(34), 1-

184.

Megnin-Viggars, O., Symington, I., Howard, L., Pilling, S. (2015). Experience of care for mental

health problems in the antenatal or postnatal period for women in the UK: A systematic

review and meta-synthesis of qualitative research. Archive of Women’s Mental Health, 18,

745-759.

Melnyk, B., Fineout-Overholt, E., Gallagher-Ford, L., Kaplan, L. (2012). The state of evidence-

based practice in US nurses. The Journal of Nursing Administration, 42(9), 410-417.

Melnyk, B. (2012). Achieving a high reliability organization through implementation of the

ARCC model for systemwide sustainability of evidence-based practice. Nursing

Administration Quarterly, 36(2), 127-135.

Mitchell, A., Selmes, T. (2007). Why don’t patients attend their appointments? Maintaining

engagement with psychiatric services. Advances in Psychiatric Treatment, 13(6), 423-434.

Moehler, E., Brunner, R., Wiebel, A., Reck, C., Resch, F. (2006). Maternal depressive


51

symptoms in the postnatal period are associated with long-term impairment of

mother–child bonding. Archives of Women’s Mental Health, 9(5), 273-278.

National Alliance on Mental Illness (2018). Mental Health Facts in America. Retrieved from:

https://www.nami.org/NAMI/media/NAMI-Media/Infographics/GeneralMHFacts.pdf

National Institute of Mental Health, (2018). Statistics. Retrieved from:

https://www.nimh.nih.gov/health/statistics/index.shtml#Intro

Office of Disease Prevention and Health Promotion (2018). Social Determinants of Health.

Retrieved from: https://www.healthypeople.gov/2020/topics-objectives/topic/social-

determinants-of-health

Parikh, A., Gupta, K., Wilson, A. C., Fields, K., Cosgrove, N. M., & Kostis, J. B. (2010). The

effectiveness of outpatient appointment reminder systems in reducing no-show rates. The

American journal of medicine, 123(6), 542-548.

Percac-Lima, S., Singer, D., Cronin, P., Chang, Y., Zai, A. (2016). Can text messages improve

attendance to primary care appointments in underserved populations? Journal of Health

Care for the Poor and Underserved, 27(4), 1709-1725.

The Pell Institute (2018). Using a Logic Model. Retrieved from:

http://toolkit.pellinstitute.org/evaluation-guide/plan-budget/using-a-logic-model/

Poston, R., Haney, T., Kott, K., & Rutledge, C. (2017). Interprofessional team performance,

optimized. Nursing management, 48(7), 36-43.

Robotham, D., Satkunanathan, S., Reynolds, J., Stahl, D., Wykes, T. (2016). Using digital

notifications to improve attendance in clinic: Systematic review and meta-analysis. British

Medical Journal Open, doi: 10.11136/bmjopen-2016-012116


52

Scientific American, Editors, (2012). The Neglect of Mental Illness Exacts a Huge Toll,

Human and Economic. Retrieved from: https://www.scientificamerican.com/article/a-

neglect-of-mental-illness/

Shah, et al., (2016). Targeted reminder phone calls to patients at high-risk of no-show for

primary care appointment: A randomized trial. Journal of General Internal Medicine,

31(12), 1460-1466.

Teo, A., Forsberg, C., Marsh, H., Saha, S., Dobscha, S. (2017). No-show rates when phone

appointment reminders are not directly delivered. Psychiatric Services, doi:

10.1176/appi.ps.201700128.

Tofighi, B., Grazioli, F., Bereket, S., Grossman, E., Aphinyanaphongs, Y., Lee, J. (2017). Text

message reminders for improving patient appointment adherence in an office-based

Buprenorphine program: A feasibility study. The American Journal on Addictions, 26, 581-

586.

van Eck van der Sluijs, J. F., Castelijns, H., Eijsbroek, V., Rijnders, C. A., van Marwijk, H. W.

J., & van der Feltz-Cornelis, C. M. (2018). Illness burden and physical outcomes associated

with collaborative care in patients with comorbid depressive disorder in chronic medical

conditions: A systematic review and meta-analysis. General Hospital Psychiatry:

Psychiatry, Medicine and Primary Care, 50, 1-14.

Weinreb, L., Byatt, N., Moore-Simas, T., Tenner, K., Savageau, J. (2014). What happens

to mental health treatment during pregnancy? Psychiatry Quarterly, 85(3), 349-355.

World Health Organization (2017). Depression fact sheet. Retrieved from:

http://www.who.int/mental_health/prevention/genderwomen/en

Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). Program evaluation

standards. Thousand Oaks, CA: Sage


53

Zimmerman, S. (2015). Insight: Tackling healthcare’s costly problem of missed appointments.

Healthcare Finance. Retrieved from: http://www.healthcarefinancenews.com/blog/insight-

tackling-healthcares-costly-problem-missed-appointments


54

Appendix A

Table of Evidence

Citation Research Design/Level of Evidence

Sampling & Methods

Data Analysis Findings Comments/Limitations

Boksmati, N., Butler-Henderson, K., Anderson, K., Sahama, T. (2016). The effectiveness of SMS reminders on appointment attendance: A meta-analysis. Journal of Medical Systems, doi: 10.1007/s10916-016-0452-2

Level Ib: Experimental Design. Meta-Analysis of RCT and other study designs

Literature review of OvidSP, PubMed, EMBASE and CINAHL with keywords texting, text reminder, text messaging, reminder systems. All articles evaluated SMS system against a control group of no reminder, English language, published between 2005-2015. 303 records found, 28 included in analysis. 13 (46%) RCT, 15

Frequency analysis for country, attendance rate, specialty, SMS timing, study design. Attendance rate was primary outcome. 95% confidence interval, intervention group compared to control group.

SMS reminders improved appointment attendance compared to control group with OR of 1.62. 75% of SMS sent within 48 hours of appointment 75% of settings sent only one reminder, No differences found for age, timing of reminder, or if reminder was personal or generic.

4 studies conducted in United States, majority conducted in the UK.


55

(53%) observational.

Callinan, C. Kenney, B., Hark, L., Murchison, A., Dai, Y., Leiby, B., Mayro, E., Bilson, J., Haller, J. (2015). Improving follow-up adherence in a primary eye care setting. American Journal of Medical Quality, 32(1), 73-79.

Level Ic: Experimental Design. Randomized controlled trial

Prospective, single-blinded RCT. Patients in Philadelphia eye clinic, randomized to usual care (UC), automated intervention, personal intervention. UC (N:364) received auto call reminder day before appointment, automated intervention (N: 365) received mailed appointment reminder, day prior reminder call and follow up postcard if they missed appointment, personal intervention (N:

Primary outcome measure was adherence to follow-up appointments. 3-month time period, 80% power to detect increase in attendance in either intervention compared to UC. 95% CI, P value of 0.5, Poisson regression model to detect treatment group differences.

Personal intervention more likely to attend appointments (38%) than UC (28%) and automated intervention (30%). No significant difference between UC and automated.

Urban setting, sample primarily African American, primarily Medicaid/Medicare coverage. Maintaining up-to-date patient information was listed as challenge to providing reminders.


56

366) all of the above plus personal telephone call prior to appointment and if appointment was missed.

Clouse, K., Williams, K., Harmon, J. (2015). Improving the no-show rate of new patients in outpatient psychiatric practice: An advanced practice nurse-initiated telephone engagement protocol quality improvement project. Perspectives in Psychiatric Care, 53, 127-134.

Level IIIe: Observational, analytic design. Convenience sample

Convenience sample of 27 subjects who met criteria, 18 of which consented to participate. 78% female, no formal assessment tools, TEP flowchart used to document each contact, intervention is two patient contacts – brief call which described first visit, personal reminder call day before appointment.

Flow-sheets were used to document each patient contact.

80% of patients who received both contacts attended appointment, attendance rates better for those who received both contacts, overall 7% improvement in show rates for intervention compared to UC. No effect for age or type of insurance.

Small sample, no control group, maintaining up-to-date patient information was listed as a challenge to providing reminders.


57

Crutchfield, T., Kistler, C. (2017). Getting patients in the door: Medical appointment reminder preferences. Patient Preference and Adherence, 11, 141-150.

Level IVb: Observational/Descriptive Convenience sample - survey

Participants recruited via listserv of university (N=251). Participants completed online survey inquiring about appointment reminder preferences.

Used statistical software for descriptive analyses of which reminder methods were preferred.

Email, phone and text were preferred reminder methods, within one to six days prior to appointment, single reminder was preferred.

Selection bias – mostly Caucasian, highly educated sample.

Finkelstein, S., Liu, N., Jani, B., Rosenthal, D., Poghosyan, L. (2013). Appointment reminder systems and patient preferences: Patient technology usage and familiarity with other service providers as predictive variables. Health Informatics Journal, 19(2), 79-90.

Level IVd: Observational/Descriptive Cross-sectional survey

Assessed patient preference for five different reminder systems. N= 161, urban setting, diverse sample, low-income, convenience sample.

Self-report survey instrument, rank ordered reminder preference from most to least preferred, assessed patient responsiveness to various appointment reminder systems. Descriptive statistics and frequencies were analyzed using SPSS, Friedman nonparametric to assess ranked preferences and Wilcoxon Signed Ranks to assess for significant differences between

Preferences were ranked as: cell phone, home phone, text message, email, direct mail. Familiarity with and usage of technology were indicators of responsiveness to reminders.

78% of sample was female, urban setting, majority Medicaid/Medicare. Self-report convenience sample.


58

ranks. Effectiveness of and responsiveness to reminder systems were dependent variables.

Gurol-Urganci, I., de Jongh, T., Vodopivec-Jamsek, V., Atun, R., Car, J. (2013). Mobile phone messaging reminders for attendance at healthcare appointments (Review). Cochrane Database of Systematic Reviews, doi: 10.1002/14651858.cd007458.pub3

Level Ia: Experimental Design. Systematic review of RCTs

Cochrane review of MEDLINE, EMBASE, PsycINFO, CINAHL and grey literature. RCTs assessing text messaging as reminder system. Included 8 RCTs with total of 6,615 participants.

GRADEpro software used to assess quality of evidence, RR used for effect measures, meta-analysis to calculate overall effect size.

Moderate quality evidence that text reminders improve appointment adherence compared to no reminders (RR: 1.14) 95% CI. Three studies found text and phone reminders equal. Overall 78% attendance for text, 80% attendance for phone call, 68% attendance for no reminder.

No conclusions on timing of messages or combining reminder methods.

Kannisto, K., Koivunen, M., Valimaki, M. (2014). Use of mobile phone text message reminders in health care services: A narrative literature review. Journal of Medical Internet Research, 16(10), doi: 10.2196/jmir.3442

Level Ib: Experimental Design. Systematic review of RCT and other study designs

Systematic review of PubMed, CINAHL, PsycINFO and other databases, original yield of 911 articles, 60

Identified 14 variables of interest, entered into data extraction grid, quality appraised using Mixed Method Appraisal Tool.

Outcomes improved in 77% of studies, with 11 of those studies investigating appointment attendance.

Diverse methodology and varying quality in studies, frequently changing patient contact information identified as challenge.


59

included in review. 73% SMS only, 27% SMS/Plus. 35% RCT, 95% quantitative design. Sample size from 4 - 9,959. Looked at various health reminders, not just appointment

Lin, C., Mistry, N., Bonch, J., Li, H., Lazebnik, R. (2016). Text message reminders increase appointment adherence in a pediatric clinic: A randomized controlled trial. International Journal of Pediatrics, doi: 10.1155/2016/8487378

Level Ic: Experimental Design Randomized Controlled Trial/Blinded Assessment

RCT in urban pediatric clinic. Randomized to receive text or not receive text in addition to UC of phone reminder. N = 170. Text sent three days prior to appointment, charts reviewed to determine who attended appointments.

Sampled for 90% power to detect 15% difference between groups, 169 participants were randomized into control or intervention group, 95% CI with p value >0.05 as significant.

Significant difference in no-show rate for control group (38.1%) compared to 23.5% for intervention group, intervention group more likely to keep, cancel or reschedule appointments.

Urban setting, no-show rate remained 24% even with text message.

McLean, S., Gee, M., Booth, A., Salway, S., Nancarrow, S., Cobb, M., Bhanbhro, S. (2014).

Level Ib: Experimental Design.

3 interrelated systematic reviews of

Identified six variables of interest on patient

Timing of reminders from 1-7 days has

Weak evidence that reminder plus (health information in reminder


60

Targeting the use of reminders and notifications for uptake populations (TURNUP): A systematic review and evidence synthesis. Health Services and Delivery Research, 2(34), doi: 10.3310/hsdr02340

Systematic review of RCT and other study designs

reminder systems, reviewed Cochrane Library, EMBASE and others. 31 RCTs and 11 other systematic reviews were included.

attendance, data extraction tool used to determine patterns and quality of evidence.

same outcome, reminders increase cancellations, to optimize reminders patient information must be up-to-date, information beyond date, time and location is helpful for first appointments.

or dual reminders) improves adherence.

Molfenter, T. (2013). Reducing appointment no-shows: Going from theory to practice. Journal of Substance Use and Misuse, 48(9), 743-749

Level IIc: Quasi-Experimental Design Prospectively controlled study

SAMHSA funded program investigating 67 treatment agencies across 10 states with focus on improving appointment adherence.

Programs implemented practices of their choosing thought to reduce no-show rates, ANOVA analysis separated sites by urban and rural and differences between initial and post no-show rates with descriptive statistics and frequencies for initial appointment and follow-up no-shows.

Significant difference in mean no-show rates post intervention: 16.48%, Reminder calls did improve appointment attendance though it was not a significant finding in this study, creating a welcoming environment, reduced wait times, more capacity and behavioral

Appointment reminders alone may not be adequate.


61

engagement were most significant interventions.

Parikh, A., Gupta, K., Wilson, A. C., Fields, K., Cosgrove, N. M., & Kostis, J. B. (2010). The effectiveness of outpatient appointment reminder systems in reducing no-show rates. The American journal of medicine, 123(6), 542-548.

Level 2c: Prospective, randomized, parallel design clinical trial.

Random assignment to personal telephone reminder, automated telephone reminder, no reminder. Reminders made three days prior to appointment with continued attempts for following two days if no contact made. N= approx. 3,000 per group based on 80% power.

No-show rate was primary outcome of interest. Analyzed using Intention to Treat analysis, chi-square analysis to determine between group differences using OR and 95% CI.

Significantly higher no-show rate for new patients compared to established patients across all groups (17.7%, 15.9%), No-show rates significantly lower for personal (13.6%), auto (17.3) reminders compared to no-reminder (23.1%).

Patients found reminders helpful but did not remember if reminder was personal or automated.

Percac-Lima, S., Singer, D., Cronin, P., Chang, Y., Zai, A. (2016). Can text messages improve attendance to primary care appointments in underserved populations? Journal of Health Care for the Poor and Underserved, 27(4), 1709-1725.

Level Id: Experimental Design Pseudo RCT

Appointment reminder intervention outside of Boston, MA. Randomized into Intervention = 8,425, Control = 2,679. Intervention

Chi-square analysis was used for between group differences, no show rates compared using logistic regression, 88% power.

With Intention to Treat group, no significant difference was found (due to many subjects not having working mobile phones/not receiving

Phone access and knowledge of how to use text message was a factor. Inaccurate patient contact information had substantial impact on reminder outreach.


62

group received 2 texts (7 days before and 1 day before) plus phone call, Control group received only phone call 2 days before.

messages). In the matched control analysis, intervention had 32% reduction in missed appointments.

Robotham, D., Satkunanathan, S., Reynolds, J., Stahl, D., Wykes, T. (2016). Using digital notifications to improve attendance in clinic: Systematic review and meta-analysis. British Medical Journal Open, doi: 10.11136/bmjopen-2016-012116

Level Ib: Experimental Design. Systematic review of RCT and other study designs

Search of MEDLINE, EMBASE, PsycINFO, Cochrane Library. 26 articles were included. Text Intervention. N= 8,345 Control N= 7,731.

Primary outcome was appointment adherence, meta-analysis determined pooled effect size for intervention and control, 95% CI.

Intervention group was 23% more likely to attend appointments and 25% less likely to no-show. RR: 1.23 & 0.75. Two or more notifications increased attendance by 19%, reminders increased cancellations.

Findings for mental health clinics had same outcomes.

Shah, et al., (2016). Targeted reminder phone calls to patients at high-risk of no-show for primary care appointment: A randomized trial. Journal of General Internal Medicine, 31(12), 1460-1466.

Level Ic: Experimental Design Randomized Controlled Trial

2,247 patients determined at risk for appointment no-show randomized into intervention (N=1,129) or control (N=

Between group rates calculated using Absolute Risk difference. T-tests used to determine reimbursement differences between two groups, 95% CI.

No-show rate in intervention arm (22%) was significantly lower than control group (29%). Arrival, cancelation and reschedule rates

Two reminders may be better than one. Only telephone reminders, no SMS.


63

1,118). Primary outcome was no-show rate. Intervention group received telephone reminder 7 days prior to appointment. Both groups receive reminder call 3 days prior to appointment.

were higher in intervention though not statistically significant.

Teo, A., Forsberg, C., Marsh, H., Saha, S., Dobscha, S. (2017). No-show rates when phone appointment reminders are not directly delivered. Psychiatric Services, 68(11), 1098-1100.

Level IIId: Observational-Analytic Design. Case Controlled Study

Conducted within Dept. of Veteran’s Affairs, participants screened positive for depression. All participants received reminder call Friday before appointment and day before appointment. Type of contact (live, message, no-answer) was logged. N=250

Chi-square analysis used to compare reminder call and appointment attendance.

Live reminders had significantly lower no-show rate (3%) than message (24%) and no reminder (39%). Difference between message and no reminder also significant.

Notes text is becoming more popular, a two-way system that acknowledges receipt may be useful


64

Tofighi, B., Grazioli, F., Bereket, S., Grossman, E., Aphinyanaphongs, Y., Lee, J. (2017). Text message reminders for improving patient appointment adherence in an office-based Buprenorphine program: A feasibility study. The American Journal on Addictions, 26, 581-586.

Level IIId: Observational Analytic Design. Case Controlled Study

Convenience sample of opiate dependent adults. Non-random, all screened for having a mobile phone, training in using phone was offered, phone numbers were verified. N=93

Text reminders were sent 7,4,1 day prior to appointments. 16-item survey related to technology acceptance was completed afterwards to determine their preferences and opinions. Answers were coded and agreed upon. Show rates were not measured.

91% preferred text message over phone reminders, two was preferred number of reminders, receipt the week prior to appointment was preferred, age was not a factor.

Did not report on adherence/no-show rates.


65

Appendix B

Interview Protocol

Health Center Evaluation of Appointment Reminder System Process for Psychiatric Appointments

Scheduled within OBGYN/Centering Programs

Meeting Interview Protocol

Part 1: (2:15-2:25)

A. Introduction: Hello. Thank you for coming to this meeting. My name is Lacy Clayton and I am a Doctor of Nursing Practice (DNP) candidate from Drexel University. I am working to identify possible ways to reduce no-show rates for psychiatric appointments scheduled within the OBGYN or Centering clinics. Each of you were identified as someone who has expertise in this area. You will find a copy of the agenda in front of you. This also contains contact information for myself and Judy Faust. Your participation is voluntary and there will be no penalty for not participating. The information received today will be consolidated and synthesized and none of the comments will be attached to any particular person. The aim of this meeting is to gain a complete understanding of how the reminder system is intended to be delivered compared to how it is actually delivered and to get your feedback about challenges and recommendations for improving the appointment reminder system.

B. Please briefly introduce yourself and describe your role, including your specific role in the appointment scheduling and/or reminder process

Part 2: (2:25-3:25)

C. Please describe how the appointment reminder system is intended to be delivered.

a. Probe for: Is there a specific protocol? b. What is the method – phone/text/mail/automated/personal? c. How many reminders are given for one appointment? d. When are the reminder(s) sent? e. Is there a protocol for updating patient contact information?

D. Please describe how the appointment system is actually delivered. a. Probe for: What is the method – phone/text/mail/automated/personal? b. How many reminders are given for one appointment? c. When are the reminder(s) sent? d. How is patient contact information updated?


66

e. What are the main ways the intended process does not match the delivered process?

i. Why? Part 3: (3:25-3:40)

E. What are the main challenges experienced when trying to implement/deliver the appointment reminders?

a. Probe as needed

F. What are your suggestions for improving the appointment reminder system? a. Probe as needed

Part 4: (3:40-3:55) G: Review findings and confirm/clarify the appointment reminder process. Part 5: (3:55-4:00)

H. Wrap Up and Thank You


67

Appendix C

Executive Summary

Evaluation of Appointment Reminder System for Perinatal Psychiatric Appointments EXECUTIVE SUMMARY – August, 2018 The high appointment no-show rate for psychiatric appointments scheduled within the Obstetrics Department was identified by staff as a practice area of concern. Patients scheduled for psychiatry appointments within the obstetrics clinic have a no-show rate of 58%. Patients scheduled within the Centering Pregnancy/Parenting program have a no-show rate of 61%. The total number of no-shows from January to August, 2018 is 274. Despite this high no-show rate, psychiatry is the most requested referral within this department. Though missed appointments are experienced across all medical specialties, evidence suggests the rate of psychiatric missed appointments may be twice that of other specialties (Mitchell & Selmes, 2007). Research supports that perinatal women are particularly at risk for missing scheduled appointments due to logistics such as transportation and childcare (Grote et al., 2015). While multiple factors contribute to missed appointments, appointment reminders are proven effective for increasing appointment attendance. An evaluation of the appointment reminder system specific to psychiatry appointments within the obstetrics department was conducted. Five key informants from obstetrics, centering pregnancy, psychiatry, and the call center convened to provide background on the current appointment reminder system. The objective was to determine if the current reminder system aligns with evidence-based practice. Recommendations for improving the appointment reminder system with the goal of increasing appointment attendance are provided below. When perinatal women miss their scheduled psychiatric appointments, they are not accessing the care they need. Depression is the leading cause of disability for persons age 15-44 in the United States (National Institute of Mental Health, 2018). Perinatal women are particularly vulnerable as it relates to mental health with one in seven women receiving a diagnosis of postpartum depression (American Psychiatric Association, 2017) compared to one in 10 women in the general population (Centers for Disease Control, 2017). As few as 35% of pregnant women with a documented history of depression receive treatment. The main predictor for the duration of a postpartum depression episode is the amount of time lapsing before treatment is initiated, making early detection and treatment crucial for these women and their families (Hansotte, Payne & Babich, 2017).

SIGNIFICANCE

BACKGROUND


68

From a financial perspective, missed appointments are costly. Estimates report that each missed appointment costs an organization $200 (Kheirkhah et al., 2015). Based on this appraisal, missed perinatal psychiatric appointments could potentially cost the obstetrics department over $80,000 annually. Evidence suggests that two appointment reminders within seven days of the appointment are preferred (Robotham et al., 2016). Personal reminders are also shown to improve attendance over automated systems (Teo, Forsberg, Marsh, Saha and Dobscha, 2017). Inaccurate patient contact information is repeatedly found to impede the effectiveness of appointment reminder systems (Percac-Lima et al., 2016). Key findings from the evaluation of the perinatal psychiatry appointment reminder system:

v Patients receive one automated Televox reminder 48 hours before appointment v Centering patients receive additional, personalized reminders from their treatment team v Patient contact information is not routinely updated v The daily Televox report is not being reviewed, resulting in canceled appointments on Televox

report not being noted or offered to waiting patients

The above findings offer opportunity for improvement. The daily Televox report not being reviewed for cancelations is especially noteworthy. This results in available appointment times remaining unfilled, inefficient use of provider time, and an artificial inflation of the no-show rate. Findings from this evaluation offer recommendations for improving the appointment reminder system within the perinatal psychiatric service. Suggested changes have the potential to increase appointment attendance rates thereby increasing patient access to care. The fiscal outcomes of the department may also improve. Appointment attendance rates may be increased by:

v Adding a second Televox reminder within 7 days of the appointment v Implementing a strategy to routinely update patient contact information v Assigning staff to review the daily Televox appointment report v Noting appointment cancelations and scheduling waiting patients into those openings

The institution can employ a rapid cycle improvement strategy to determine if these suggested interventions improve appointment attendance. Scheduling staff can implement the above suggestions over a four-week period, reviewing the no-show rate for that span of time. If the no-show rate is reduced, it may be worthwhile to permanently incorporate these suggestions into the appointment reminder system.

KEY LEARNING

RECOMMENDATIONS


69

American Psychological Association (2017). What is postpartum depression and anxiety? Retrieved from: http://www.apa.org/pi/women/resources/reports/postpartum- depression.aspx Center for Disease Control (2017). Depression among women. Retrieved from: https://www.cdc.gov/reproductivehealth/depression/index.htm Grote, N., Katon, W., Russo, J., Lohr, M.J., Curran, M., Galvin, E., Carson, K. (2015). Collaborative care for perinatal depression in socioeconomically disadvantaged women: A randomized controlled trial. Depression and Anxiety, 32, 821-834. Hansotte, E., Payne, S., Babich, S. (2017). Positive postpartum depression screening practices and subsequent mental health treatment for low-income women in Western countries: A systematic literature review. Public Health Reviews, 38(3), doi: 10.1186/s40985-017-005 Kheirkhah, P., Feng, Q., Travis, L. M., Tavakoli-Tabasi, S., & Sharafkhaneh, A. (2015). Prevalence, predictors and economic consequences of no-shows. BMC health services research, 16(1), 13. Retrieved from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4714455/pdf/12913_2015_Article_1243.pdf Mitchell, A., Selmes, T. (2007). Why don’t patients attend their appointments? Maintaining engagement with psychiatric services. Advances in Psychiatric Treatment, 13(6), 423-434. National Institute of Mental Health, (2018). Statistics. Retrieved from: https://www.nimh.nih.gov/health/statistics/index.shtml#Intro Percac-Lima, S., Singer, D., Cronin, P., Chang, Y., Zai, A. (2016). Can text messages improve attendance to primary care appointments in underserved populations? Journal of Health Care for the Poor and Underserved, 27(4), 1709-1725. Robotham, D., Satkunanathan, S., Reynolds, J., Stahl, D., Wykes, T. (2016). Using digital notifications to improve attendance in clinic: Systematic review and meta-analysis. British Medical Journal Open, doi: 10.11136/bmjopen-2016-012116 Teo, A., Forsberg, C., Marsh, H., Saha, S., Dobscha, S. (2017). No-show rates when phone appointment reminders are not directly delivered. Psychiatric Services, doi: 10.1176/appi.ps.201700128

REFERENCES