Embed Size (px)
Citation preview
Predictors and Outcomes of Nurse Practitioner Burnout in Primary Care Practices
Cilgy M. Abraham
Submitted in partial fulfillment of the requirements for the degree of
Doctor of Philosophy under the Executive Committee
of the Graduate School of Arts and Sciences
COLUMBIA UNIVERSITY
2020
© 2020
Cilgy M. Abraham All Rights Reserved
Abstract
Predictors and Outcomes of Nurse Practitioner Burnout in Primary Care Practices
Cilgy M. Abraham
Burnout among primary care providers, which include physicians, nurse practitioners,
and physician assistants, can negatively impact patients, providers, and organizations.
Researchers have reported that up to 37% of primary care physicians experience burnout, yet the
prevalence, predictors, and outcomes associated with primary care nurse practitioner burnout
remains unknown. Since 69% of nurse practitioners provide primary care to patients, this
dissertation investigates the predictors and outcomes associated with primary care nurse
practitioner burnout. A history of burnout as well as the importance of investigating burnout
among primary care nurse practitioners are discussed in the first chapter. A systematic review of
the predictors and outcomes of primary care provider burnout is discussed in the second chapter.
The third chapter describes a cross-sectional study conducted among 396 primary care nurse
practitioners from New Jersey and Pennsylvania, which investigated whether the practice
environment is associated with nurse practitioner burnout. The fourth chapter describes a cross-
sectional study investigating whether the use of multifunctional electronic health records is
associated with primary care nurse practitioner burnout. The fifth chapter includes another cross-
sectional study examining the relationship between primary care nurse practitioner burnout and
quality of care, and if the practice environment moderates the relationship between burnout and
quality of care. Finally, the sixth concluding chapter summarizes the findings from chapters two
to five and provides recommendations for future research, practice, and policy.
i
Table of Contents
List of Tables .................................................................................................................................. v
List of Figures ................................................................................................................................. v
Funding .......................................................................................................................................... vi
Acknowledgments ......................................................................................................................... vii
Dedication ....................................................................................................................................... x
Chapter 1: Introduction ................................................................................................................... 1
1.1 History of Burnout .......................................................................................................... 1
1.11 Clinician Burnout in Acute Care Settings ........................................................................... 2
1.12 Primary care and Nurse Practitioners ................................................................................. 3
1.13 Primary care Provider Burnout ........................................................................................... 5
1.14 Aims of Dissertation ........................................................................................................... 7
1.15 Gaps in the literature ........................................................................................................... 7
1.16 Conceptual Framework ....................................................................................................... 9
1.17 Study Designs ................................................................................................................... 11
1.18 IRB Application ................................................................................................................ 12
1.19 Conclusion ........................................................................................................................ 12
1.20 Tables and Figures in Chapter 1 ....................................................................................... 13
Chapter 2: Predictors and Outcomes of Burnout among Primary Care Providers in the U.S.: A
Systematic Review ........................................................................................................................ 15
2.1 Introduction ......................................................................................................................... 16
2.11 New Contributions ............................................................................................................ 18
ii
2.12 Conceptual Framework ..................................................................................................... 18
2.13 Methods............................................................................................................................. 19
2.14 Results ............................................................................................................................... 22
2.15 Discussion ......................................................................................................................... 29
2.16 Implications for Clinical Practice and Research ............................................................... 34
2.17 Limitations ........................................................................................................................ 35
2.18 Conclusion ........................................................................................................................ 35
2.19 Figure 1 ............................................................................................................................. 37
2.20 Table 1 .............................................................................................................................. 38
2.21 Table 2 .............................................................................................................................. 48
2.22 Table 3 .............................................................................................................................. 49
2.23 Table 4 .............................................................................................................................. 50
2.24 Table 5 .............................................................................................................................. 51
Chapter 3: Primary Care Practice Environment and Burnout Among Nurse Practitioners .......... 52
3.1 Introduction ......................................................................................................................... 53
3.11 Methods............................................................................................................................. 55
3.12 Results ............................................................................................................................... 60
3.13 Discussion ......................................................................................................................... 61
3.14 Conclusion ........................................................................................................................ 64
3.15 Figure 1 ............................................................................................................................. 65
3.16 Table 1 .............................................................................................................................. 66
3.17 Table 2 .............................................................................................................................. 68
iii
Chapter 4: Use of Multifunctional Electronic Health Records and Burnout among Primary Care
Nurse Practitioners ........................................................................................................................ 69
4.1 Introduction ......................................................................................................................... 70
4.11 Methods............................................................................................................................. 72
4.12 Results ............................................................................................................................... 78
4.13 Discussion ......................................................................................................................... 79
4.14 Conclusion ........................................................................................................................ 82
4.15 Figure 1 ............................................................................................................................. 83
4.16 Table 1 .............................................................................................................................. 84
4.17 Table 2 .............................................................................................................................. 86
Chapter 5: Primary care Nurse Practitioner Burnout and Quality of Care ................................... 87
5.1 Introduction ......................................................................................................................... 88
5.11 Methods............................................................................................................................. 90
5.12 Results ............................................................................................................................... 96
5.13 Discussion ......................................................................................................................... 98
5.14 Conclusion ...................................................................................................................... 101
5.15 Figure 1 ........................................................................................................................... 102
5.16 Figure 2 ........................................................................................................................... 103
5.17 Table 1 ............................................................................................................................ 104
5.18 Table 2 ............................................................................................................................ 106
5.19 Table 3 ............................................................................................................................ 107
5.20 Table 4 ............................................................................................................................ 108
Chapter 6: Recommendations for Research, Practice, and Policy .............................................. 109
iv
6.1 Discussion of Findings ...................................................................................................... 110
6.11 Strengths and Limitations ............................................................................................... 112
6.12 Recommendations for Research, Practice, and Policy .................................................... 113
Conclusion .................................................................................................................................. 116
References ................................................................................................................................... 117
Appendix A: Search Strategy for Study 1 ................................................................................... 142
v
List of Tables
Tables and Figures in Chapter 1………………………………………………………………...13 2.20 Table 1……………………………………………………………………………………..38 2.21 Table 2……………………………………………………………………………………..48 2.22 Table 3……………………………………………………………………………………..49 2.23 Table 4…………………………………………………………………………………..…50 2.24 Table 5………………………………………………………………………………..……51 3.16 Table 1…………………………………………………………………………………..…66 3.17 Table 2……………………………………………………………………………………..68 4.16 Table 1…………………………………………………………………………………..…84 4.17 Table 2………………………………………………………………………………..……86 5.17 Table 1……………………………………………………………………………………104 5.18 Table 2……………………………………………………………………………………106 5.19 Table 3……………………………………………………………………………………107 5.20 Table 4……………………………………………………………………………………108
List of Figures
2.19 Figure 1…………………………………………………………………………………..…37 3.15 Figure 1…………………………………………………………………………………..…65 4.15 Figure 1……………………………………………………………………………………..83 5.15 Figure 1…………………………………………………………………………………....102 5.16 Figure 2……………………………………………………………………………………103
vi
Funding
This dissertation was supported by grant number R36HS027290 from the Agency for
Healthcare Research and Quality, as well as from the Robert Wood Johnson Foundation-Future
of Nursing Scholars Program. The data that were used in studies two through four came from the
parent study, grant number R01MD011514, funded by the National Institute on Minority Health
and Health Disparities. The content is solely the responsibility of the authors and does not
necessarily represent the official views of the Agency for Healthcare Research and Quality, the
Robert Wood Johnson Foundation, nor the National Institute on Minority Health and Health
Disparities.
vii
Acknowledgments
This dissertation would not have been possible without the encouragement from so many
people who may not all be listed. I thank Dr. Poghosyan, dissertation sponsor, for her feedback
with improving this dissertation, allowing me to use the data from her parent study, and helping
me to become a better writer- a skill that I will use throughout my career. I also thank
dissertation committee members- Drs Liu, Stone, Topaz, and Martsolf. Specifically, I thank Dr.
Liu for frequently sharing her time, sometimes meeting with me on a weekly basis, to help me
improve my quantitative skills. I thank Dr. Stone for teaching me how to conduct a cost-effective
analysis and for opening opportunities for me during the past three years. I thank Dr. Topaz for
being a co-mentor on my R36 dissertation grant and for his valuable feedback. Lastly, I thank
Dr. Martsolf for his feedback and for being part of this dissertation committee.
Also, I sincerely thank my PhD cohort- Anthony, Katherine, Sabrina, Jiyoun, and Kelsea-
for their encouragement, kindness, support, and amazing sense of humor. Moreover, I am
grateful to the faculty and staff (past and present) at Columbia University School of Nursing. I
especially thank Dr. Norful for her mentorship, guidance, and faith in my abilities to persevere. I
thank Dr. Ghaffari for being generous with his time by helping me with STATA and for making
me a better health services researcher. I thank Dr. Smaldone for supporting me in seeking
opportunities that are aligned with my interests. I thank Judith and Ashley for their kindness and
willingness to help. I thank Drs Carter, Hessels, Arcia, Sun, and Shang for providing
constructive feedback to help me become a better scholar. I especially thank Drs Dorritie, Voigt,
and Beauchemin for all of their words of wisdom, support, and guidance. I sincerely thank my
fifth-floor workstation colleagues- Glenny, Elizabeth, & Rudy- for inspiring me, making me
laugh, and for always brightening my day. I also thank the many faculty and staff on the fifth
viii
floor, as well as from the Columbia Health Sciences library, the Operations team, the
Custodial/Janitorial staff, and the Grants Management team. Furthermore, I thank faculty and
staff at the Robert Wood Johnson Foundation- Future of Nursing Scholars program for their
support and encouragement. I also thank Dr. Chanlongbutra and the AHRQ for supporting my
dissertation.
I am incredibly grateful to my Rutgers family. In particular, I thank Dr. Pamela de
Cordova and Dr. Sarah Kelly who have had a profound influence on me and have changed my
life in more ways than I can describe. You have both seen me evolve from a teenager to now an
adult, and have been there for me every step of the way. Your compassion, patience, time,
encouragement, and kindness have shaped me into the scholar who I am today and have helped
me to see the type of person I want to become in the future. I thank you for giving me the best
foundation necessary for grad school, and for creating a safe and welcoming environment for me
to grow, listening to all of my aspirations, never doubting my abilities, and never discouraging
me from pursuing my interests. I also thank April Ancheta for inspiring me in so many ways and
for reminding me that the road less traveled is often the most fruitful. Additionally, I am grateful
to my IWL-LSP family- my wonderful cohort as well as amazing leaders- Sasha, Lisa, and Dr.
Trigg. There are no words that I can use to describe how appreciative I am to be part of the IWL
community. You all were so instrumental in developing my research skills and for preparing me
for grad school.
At age 17, Dr. Repollet told me that one day I was going to get my doctorate. At that
time, I never believed him because I could not see that in my future. However, Dr. Rep- you
were correct. I am very thankful to Ms. Hanley for her support, encouragement, and faith in my
ability to persevere. I sincerely thank Mrs. Mesa for encouraging me to be unapologetic in
ix
pursuing my dreams, advising me to listen to my heart, and reminding me to never settle for the
path of least resistance. Her inspiring words have influenced me throughout my life, and
whenever I am lost or unsure of what I should do next, Mrs. Mesa’s words have proven to be the
light in the darkness. Thank you to my peer mentor, Alma Gonzalez, who continuously inspire
me to dream big and for reminding me to make the world how it should be instead of accepting
how it is. Furthermore, I thank my cousin, Tincy, for all of her support throughout my schooling
and for being the sister that I always wanted. You have all saw something bigger in me that I
could not see in myself, and I am sincerely thankful to each and every one of you.
Most importantly, I thank my incredible family, Mom and Dad, whose continuous love
and support are the reason that I am here today. Mom- you taught me to work hard, never give-
up, believe in myself, live simply but love deeply. Dad- you taught me to think critically,
challenge myself to be better each day, be fearless, and to fight for my beliefs and convictions.
You have both instilled in me a love for learning very early on and I am forever grateful. I will
never forget the sacrifices that you have made so that I could pursue my dreams- thank you.
x
Dedication
In loving memory of Cynthia and Selbin Abraham, who continue to watch over and guide
me in all that I do.
1
Chapter 1: Introduction
1.1 History of Burnout
The investigation of burnout in health care workers began with Herbert Freudenberger in
1974. Freudenberger is credited for creating the term “burnout,” which he described as relevant
to those in the caring professions (Samra, 2018; Schaufeli, Leiter, & Maslach, 2009), such as
nursing, medicine, psychotherapy, and social work. Burnout was defined as exhaustion resulting
from demands placed on one’s energy, and is often characterized by fatigue, frustration,
cynicism, and malaise (Freudenberger, 1974; Reith, 2018).
Since then, Christina Maslach added to Freudenberger’s work by developing a multi-
dimensional model of burnout that consists of three subscales: 1) emotional exhaustion, 2)
depersonalization, and 3) personal accomplishment (Maslach & Jackson, 1981). The emotional
exhaustion subscale measures feelings of being emotionally overwhelmed and exhausted by
one’s work (Maslach & Jackson, 1981). The depersonalization subscale measures the degree of
feeling impersonal or detached toward others, whereas the personal accomplishment subscale
measures feelings of competence and success in one’s work (Maslach & Jackson, 1981). Based
on this work, the Maslach Burnout Inventory (MBI), which contains three subscales to measure
burnout, was developed. Maslach and colleagues concluded that all three subscales of the MBI
had a Cronbach’s alpha above .7 (Maslach & Jackson, 1981). Given its good psychometric
property, for several decades, the MBI has been extensively used to measure burnout among
professionals such as nurses, physicians, residents, and social workers in a variety of settings
including different hospital units (e.g., critical care/intensive care units, medical/surgical units),
long-term care settings, and ambulatory outpatient facilities (Reith, 2018; Tawfik et al., 2017;
Corboda et al., 2011; Schaufeli et al., 2009; Jennings, 2008; Schaefer & Moos, 1996).
2
1.11 Clinician Burnout in Acute Care Settings
Burnout is a national health crisis affecting health care providers in the United States
(U.S.) (Noseworthy et al., 2017). Nearly 49% of pediatric critical care physicians (Shenoi et al.,
2018), 76% of emergency medical residents (Lin et al., 2019), and 25% of maternity nurses
experience burnout (Clark & Lake, 2020). Burnout affects clinicians by impeding their ability to
deliver high quality, safe care to patients (Noseworthy et al., 2017). Clinicians experiencing
burnout are also more likely to leave their job (Willard-Grace et al., 2019), report leaving
necessary care undone (Clark & Lake, 2020), commit medical errors, report lower quality of
patient care, and have lower satisfied patients (West, Dyrbye, & Shanafelt, 2018). As a result, a
2019 landmark report identified clinician burnout as a public health crisis (Jha et al., 2019) and
the National Academy of Medicine (2018) launched an initiative to reduce clinician burnout and
improve patient and clinician well-being.
An extensive body of research exists on clinician burnout in the acute care settings (West
et al., 2016; Bragard, Dupuis, & Fleet, 2015; Dewa et al., 2017; Cimiotti et al., 2012; McHugh et
al., 2011). Several researchers report that organizational (i.e., practice environment) and
structural (e.g., use of electronic health records [EHRs] containing multifunctional features
including computerized capabilities and electronic reminders for decision support) features
within health care settings are the most common predictors of clinician burnout (Robertson et al.,
2017; Jha et al., 2019; National Academy of Medicine, 2018; Van Bogaert et al., 2013; McHugh
et al., 2011). For example, poor practice environments for hospital nurses, characterized by low
autonomy to make patient care decisions, multiple job demands, and limited support from
administrators lead to burnout (McHugh et al., 2011), and burnt-out nurses were more likely to
perceive lower quality of care delivered to patients (Nantsupawat et al., 2016; Poghosyan et al.,
3
2010). For physicians, greater job demands and high workload, use of multifunctional EHRs,
limited teamwork, long hospital shifts, and inefficient working relations with colleagues were all
associated with physician burnout (Bragard et al., 2015; West et al., 2016; Olson et al., 2018).
1.12 Primary care and Nurse Practitioners
Although there is much evidence on clinician burnout in the acute care setting, most
health care services in the U.S. are delivered in primary care practices (Petterson et al., 2018),
yet to date little is known about provider burnout in primary care. Primary care serves as the first
point of entry to the health care system and includes services such as disease prevention, health
promotion, and chronic disease care management (World Health Organization, 2019; Shi, 2012).
Access to primary care is critical for improved outcomes including lower disease and death rates,
fewer hospitalizations and emergency department utilization (Petterson et al., 2018; Shi, 2012;
Chang et al., 2011), and fewer visits to specialists among those with access to primary care
(Szafran et al., 2018). Researchers also report that an investment in primary care can result in
more than a six-fold decrease in Medicare costs for inpatient and post-acute care services
(Lazris, Roth, & Brownlee, 2018; Reschovsky et al., 2012). Lower health care costs and lower
mortality may be explained by the fact that people receiving primary care are more likely to
receive preventative health care services which can help to detect potential health problems
before they worsen and require costly intervention (World Health Organization, 2019). Thus,
primary care is central to ensuring optimal patient outcomes (World Health Organization, 2019).
In addition, primary care can be delivered in interdisciplinary teams that include
physicians, physician assistants, and Nurse Practitioners (NPs) (Petterson et al., 2018).
Traditionally, primary care was delivered by physicians (Szafran et al., 2018), however the
number of primary care physicians in the U.S. has decreased and only 20% of new physicians are
4
entering primary care (The Lancet, 2019). Furthermore, in 2014, over 400 million patient visits
were made to a primary care physician in the U.S. (Petterson et al., 2018; Centers for Disease
Control and Prevention, 2014), which shows that there is a high demand for primary care
services. However, there only exists approximately one primary care physician for nearly 1,450
people in the U.S. (Petterson et al., 2018). Given the high usage of primary care, the limited
supply of primary care physicians, and the decline in interest to practice in primary care among
graduates of medical schools in the U.S. (Petterson et al., 2018), there exists major challenges to
meeting the demands for patient care in primary care practices. However, primary care providers
(PCPs) such as NPs and physician assistants are increasingly delivering health care to patients in
a variety of primary care settings and thus are changing the primary care workforce (Petterson et
al., 2018). Specifically, of these providers, by 2030, NPs will comprise about one-third of all
PCPs (U.S. Department of Health and Human Services, 2016).
Primary care NPs are registered nurses with advance training at the Master’s or Doctoral
level and can assess, diagnose, and manage patient conditions, order and interpret tests, prescribe
medications, and provide patient education (AANP, 2020). Given their extensive training,
primary care NPs can be utilized to meet the demands for primary care. For example, one
national study found that NPs were providing health care services to patients in rural and low-
income communities, thereby increasing access to health care for these patients (Xue, Smith, &
Spetz, 2019). Another study found a 7.6% and 7.1% increase from 17.6% and 15.9% in the
number of NPs practicing in rural and non-rural areas since 2008 (Barnes et al., 2018), which
suggests an increase in the number of NPs providing patient care in underserved communities.
Additionally, NPs deliver cost-effective care (Abraham et al., 2019; Kuo et al., 2015; Martin-
Misener et al., 2015) and have the potential to reduce health disparities (Poghosyan & Carthon,
5
2017) by providing accessible, high quality care to minorities, low-income individuals, women,
the elderly, and individuals with disabilities (Buerhaus et al., 2015; Weston et al., 2018;
Buerhaus, 2018). Given the central role that NPs have in reducing health disparities and
increasing access to cost-effective health care, organizations such as the Federal Trade
Commission (2014), National Governors Association (2012), and the American Association of
Retired Persons (Jenkins, 2018) have all encouraged the use of NPs in primary care.
1.13 Primary care Provider Burnout
Although the primary care workforce is projected to increase significantly through the
use of NPs, few researchers have investigated the predictors and outcomes of burnout in the
primary care setting (Rabatin et al., 2016; Williams et al., 2007; Linzer et al., 2009). Moreover,
many of these researchers have traditionally focused on the primary care physician workforce to
the exclusion of primary care NPs and physician assistants (Rabatin et al., 2016; Babbott et al.,
2014; Williams et al., 2007; Linzer et al., 2009). Such studies show that poor practice
environments and use of EHRs with multifunctional features, reminders, and alert systems add to
provider stress, workload, job dissatisfaction (Robertson et al., 2017; Babbott et al., 2014; Linzer
et al., 2009) and lead to primary care physician burnout (Robertson et al., 2017). In addition,
researchers conducting a multi-site study found primary care physician burnout to be associated
with poor quality of patient care, which they defined poor quality of care as discharging patients
early to make care delivery feasible, not fully discussing treatment options or answering patient
questions, making treatment or medication errors, prescribing medications without fully
assessing patients, and skipping procedures to facilitate early patient discharge (Williams et al.,
2007). As a result, it appears that among primary care physicians, the practice environment and
6
use of multifunctional EHRs can contribute to their burnout which may then impact the quality
of care they deliver to patients.
Although there are more than 290,000 NPs licensed in the U.S. (AANP, 2020), and
nearly 70% of NPs deliver primary care services to patients (AANP, 2020), the prevalence,
predictors, and outcomes associated with primary care NP burnout remains minimally
investigated. This is a limitation because researchers have found that 46.1% of NPs work in a
poor practice environment (Carthon et al., 2020) and an extensive body of evidence suggests that
working in a poor or challenging practice environment is a known contributor to provider
burnout (Abraham et al., 2019; West et al., 2018; Willard-Grace et al., 2019; Rabatin et al., 2016;
Babbott et al., 2014; Edwards et al., 2018; Helfrich et al., 2014; Blechter et al., 2017; Spinelli et
al., 2016; Olayiwola et al., 2018). However, compared to other PCPs, NPs are more likely to
practice in low-income, high-minority practices (Barnes et al., 2018; Buerhaus et al., 2015)
which typically have fewer resources needed for patient care (Chan et al., 2019), and therefore
NPs may be experiencing unique challenges in their practices. Furthermore, in another study,
researchers have found that NP role is not clearly defined in some practices and some
administrators do not have a clear understanding of NP skills (Poghosyan & Aiken, 2015). These
researchers have also reported that one in every four NPs lack necessary organizational support
(Poghosyan & Aiken, 2015), which may create challenging environments to provide patient care.
As a result, these challenges might predispose NPs to burnout and ultimately impact the quality
and safety of patient care in primary care settings. Thus, it is critically important to investigate
factors associated with NP burnout in primary care because such knowledge will inform future
interventions to reduce burnout, improve provider well-being, and patient outcomes.
7
1.14 Aims of Dissertation
The overall purpose of this dissertation is to identify factors associated with primary care
NP burnout. Table 1 details the chapters of this dissertation and the aims for each dissertation
study.
1.15 Gaps in the literature
The gaps existing in the literature serve as the foundation for the four studies that
encompass this dissertation.
Study 1/Chapter 2: Existing systematic reviews have investigated burnout among health
care providers such as physicians (Panagioti et al., 2018; Wen et al., 2016), nurses (Monsalve-
Reyes et al., 2018), and physician assistants and NPs (Hoff, Carabetta, & Collinson, 2017).
However, none of these existing systematic reviews are inclusive of all PCPs (i.e., physicians,
NPs, and physician assistants) within a single review, which is a limitation because primary care
in the U.S. is delivered by an interdisciplinary team of providers including NPs, physician
assistants, and physicians (Petterson et al., 2018), and it is important to understand the factors
that contribute to their collective experiences of burnout. To our knowledge, a comprehensive
review on the predictors and outcomes of burnout among all PCPs does not exist. Therefore, this
is the first review to examine predictors and outcomes of burnout among all PCPs in the U.S.
Given the adverse consequences associated with clinician burnout, this systematic review
(chapter 2) fills a gap in the primary care literature by generating knowledge on the prevalence of
burnout in PCPs and how burnout impacts PCPs and the care that they deliver to patients.
Study 2/Chapter 3: The findings from our systematic review (study 1/chapter 2) revealed
that the primary care practice environment was a commonly reported, modifiable predictor of
PCP burnout (Helfrich et al., 2014; Willard-Grace et al., 2013). Additionally, several studies
8
included in the systematic review focused solely on primary care physician burnout (Rabatin et
al., 2016; Robertson et al., 2017; Babbott et al., 2014; Yoon, Daley, & Curlin, 2017) whereas
none of the study authors focused solely on primary care NP burnout. Instead, several authors
included all PCPs (i.e., primary care NPs, physician assistants, and physicians) (Williard-Grace
et al., 2019; Williard-Grace et al., 2013; Olayiwola et al., 2018; Edwards et al., 2017). However,
of the authors that included NPs (Edwards et al., 2018; Olayiwola et al., 2018; Willard-Grace et
al., 2019; Willard-Grace et al., 2013), they combined NPs and physician assistants as the same
group of professionals, which is a limitation because NPs and physician assistants are from two
different professions and have different training, skills, competencies, and regulations governing
their ability to practice clinically. As a result, to date, no study has examined modifiable
predictors associated with primary care NP burnout. Given this gap in the literature, study 2
(chapter 3) of this dissertation seeks to fill these gaps by identifying the prevalence of burnout
among primary care NPs and whether the practice environment is associated with NP burnout in
primary care practices.
Study 3/Chapter 4: Similarly, in the systematic review, we found that use of
multifunctional EHRs was a commonly reported, modifiable predictor of PCP burnout (Babbott
et al., 2014; Robertson et al., 2017). However, those researchers investigating PCP burnout have
mostly focused on primary care physicians (Babbott et al., 2014; Robertson et al., 2017). To our
knowledge, no researcher has published a study investigating use of multifunctional EHRs with
primary care NP burnout, which is a limitation because NPs deliver a significant portion of
primary care services in the U.S. (U.S. Department of Health and Human Services, 2016), yet
factors contributing to their burnout remain largely unknown. Thus, study 3 (chapter 4) of this
9
dissertation attempts to fill this gap in the literature by determining whether use of
multifunctional EHRs is associated with primary care NP burnout.
Study 4/Chapter 5: Most studies in our systematic review (study 1/chapter 2) did not
investigate outcomes of PCP burnout (Helfrich et al., 2014; Olayiwola et al., 2018; Robertson et
al., 2017; Babbott et al., 2014; Edwards et al., 2018; Yoon et al., 2017). Of the few studies that
did (Rabatin et al., 2016; Linzer et al., 2009; Ratanawongsa et al., 2008; Williams et al., 2007),
none have investigated outcomes specifically for primary care NP burnout, revealing another
major gap in the literature. One researcher found primary care physician burnout to be associated
with poor quality of patient care (Williams et al., 2007). Furthermore, other researchers report
that physician burnout leads to compromised patient care (i.e., poor care quality, medical errors,
and lower patient satisfaction) (West et al., 2018). Yet, to date, no researcher has investigated
whether primary care NP burnout is associated with lower quality of care, and if the practice
environment moderates that relationship. This is a major gap in the literature because primary
care NPs provide a large proportion of care to patients (U.S. Department of Health and Human
Services, 2016), but we do not know their levels of burnout nor its impact on care quality. Thus,
to fill this gap, the purpose of study 4 (chapter 5) is to investigate the association between
primary care NP burnout and quality of care, and to assess if the primary care NP practice
environment moderates the relationship between burnout and quality of care.
1.16 Conceptual Framework
This dissertation will focus on the relationship between potential predictors (i.e., practice
environment and use of multifunctional EHRs) and an outcome (i.e., quality of care) associated
with primary care NP burnout while controlling for NP and primary care practice characteristics.
Given this focus, the dissertation will be guided by the Clinician Well-Being and Resilience
10
model (Brigham et al., 2018), in which clinician and practice characteristics are taken into
consideration in predicting clinician burnout and the model also shows the relationship between
clinicians and the consequences of burnout (i.e., clinician well-being, clinician-patient
relationship, and patient well-being). The Clinician Well-Being and Resilience model was
developed by researchers at the National Academy of Medicine (Brigham et al., 2018). The
model was created because there was a lack of existing conceptual models to illustrate factors
associated with clinician burnout and well-being across all health care professions (Brigham et
al., 2018).
For this dissertation, we adapted the Clinician Well-Being and Resilience model (Figure
1) to focus on burnout and variables associated with burnout (i.e., practice environment and use
of multifunctional EHRs), and a potential outcome of burnout (i.e., lower quality of care). The
practice environment is a complex phenomenon that describes how organizational structure
within the work setting has the potential to impact an employees’ performance and productivity
(Kanter, 1976; Carthon et al., 2020; Poghosyan et al., 2013). The practice environment for
primary care NPs is defined as an NP’s perception of these four domains: NP-Physician
relations, Professional Visibility, Independent Practice and Support, and NP-Administration
Relations (Poghosyan et al., 2013). A limitation in any of these domains may create a
challenging practice environment for NPs to deliver patient care. Use of multifunctional EHRs is
defined as use of EHR systems with functions that allow for imputing patient information,
ordering medications, and providing clinical decision support (Huang, Gibson, & Terry, 2018;
Schoen et al., 2012). When EHR systems contain more of these features, they are considered to
be multifunctional (Huang et al., 2018). For this dissertation, use of multifunctional EHRs has
been conceptualized as computerized capabilities and electronic reminder systems for decision
11
support of clinical guidelines (Friedberg et al., 2009). Quality of care is conceptualized as an
NP’s self-report of the quality of care delivered in the primary care practice. The adapted
conceptual model also accounts for variables measuring NP (i.e., age, sex, education, race,
marital status, hours worked per week, workload, experience, and years employed in current
practice) and practice (i.e., size, type of practice, hours of operation) characteristics. We
hypothesized that poor practice environments and greater use of multifunctional EHRs will be
associated with primary care NP burnout and burnt-out NPs will perceive delivering lower
quality of care. We also hypothesized that the practice environment will moderate the
relationship between NP burnout and quality of care.
1.17 Study Designs
Chapter 2 of this dissertation is a systematic review of the predictors and outcomes of
burnout among PCPs in the U.S. and was published in Medical Care Research and Review.
Based on the results from the systematic review, chapter 3 is a secondary analysis of cross-
sectional survey data collected from 396 NPs in New Jersey and Pennsylvania where we tested
the association between the primary care practice environment and NP burnout. Chapter 4 is also
a secondary analysis of the same cross-sectional survey data used in chapter 3, and we
investigated the association between use of multifunctional EHRs and primary care NP burnout.
Similarly, using the knowledge that was gained from chapter 2 on outcomes of PCP burnout,
chapter 5 is a secondary analysis of the same cross-sectional survey data as used in chapters 3
and 4, in which we investigated the association between primary care NP burnout and quality of
care, and if the primary care practice environment moderates the relationship between NP
burnout and quality of care.
12
1.18 IRB Application
This dissertation work encompasses one systematic review and three studies using
secondary data obtained from the parent study (L. Poghosyan, R01MD011514). Researchers
have obtained approval from the Institutional Review Board at Columbia University to conduct
the studies in this dissertation.
1.19 Conclusion
Clinician burnout is a public health threat and affects provider well-being and patient
safety (Jha et al., 2019). This dissertation will provide timely evidence on factors associated with
NP burnout in primary care. Moreover, NPs practice in challenging environments and use EHRs
which have the potential to be improved, but knowledge of primary care NP burnout, practice
environment, EHR use, and quality of care is limited, indicating a major gap in evidence.
Knowledge on the factors associated with primary care NP burnout is needed to ensure that, in
the future, clinician and practice administrators can be aware of these factors which could be
remedied to ultimately reduce NP burnout. Since the primary care practice environment and
usability of EHRs may be modifiable, knowledge from this dissertation will be used to develop
future interventions to improve the practice environment and operability of EHRs to
subsequently reduce burnout and improve the quality of primary care delivery.
13
1.20 Tables and Figures in Chapter 1
Table 1. Dissertation Chapters and Aims
Chapter Title Aims 2 Predictors and Outcomes of
Burnout among Primary Care Providers in the United States: A Systematic Review
1. To identify the predictors and outcomes of burnout among primary care providers within the United States.
3 Primary care Practice Environment and Burnout among Nurse Practitioners
1. To describe the NP practice environment and burnout among NPs in primary care practices. 2. Investigate whether the primary care NP practice environment (i.e., NP-physician relations, NP-administration relations, independent practice and support, and professional visibility) is associated with NP burnout.
4 Use of Multifunctional Electronic Health Records and Burnout among Primary Care Nurse Practitioners
1. Determine whether use of multifunctional EHRs is associated with NP burnout in primary care practices.
5 Primary care Nurse Practitioner Burnout and Quality of Care
1. To investigate the association between NP burnout and quality of care for patients in primary care practices. 2. To assess if the primary care NP practice environment moderates the relationship between NP burnout and quality of care.
Note. NP = Nurse Practitioner; EHR = Electronic Health Record;
14
Figure 1. Adapted Clinician Well-Being and Resilience Model
Note. EHR = Electronic Health Record; NP = Nurse Practitioner;
15
Chapter 2: Predictors and Outcomes of Burnout among Primary
Care Providers in the U.S.: A Systematic Review
This chapter contains a systematic review that was conducted to investigate the predictors
and outcomes of burnout among primary care providers in the U.S. This manuscript is published
in Medical Care Research and Review. The citation is as follows:
Abraham, C.M., Zheng, K., & Poghosyan, L. (2019). Predictors and outcomes of burnout among
primary care providers in the United States: A systematic review. Medical Care Research
and Review. https://doi.org/10.1177/1077558719888427
16
2.1 Introduction
Burnout is a national health crisis affecting health care providers throughout the United
States (U.S.) (Noseworthy et al., 2017). Nearly 37% of nurses (McHugh et al., 2011), 48% of
critical care physicians (Larkin, 2018), and 69% of surgical residents (Lebares et al., 2018)
experience burnout. Burnout has been defined as a “prolonged response to chronic interpersonal
stressors” (Maslach & Leiter, 2016, p. 103) and is characterized by the presence of emotional
exhaustion, depersonalization, and feelings of low personal accomplishment (Maslach &
Jackson, 1981). Burnout influences clinicians by impeding their ability to deliver high quality,
safe care (Noseworthy et al., 2017), which subsequently leads to poor patient outcomes and
threatens patient safety. Given the high prevalence of clinician burnout, the Federation of State
Medical Boards (2018) issued a policy report on physician wellness and burnout, and the
American Public Health Association reported alarming rates of provider burnout resulting from
organizational problems and the rapidly changing practice environment (Krisberg, 2018).
Increasingly, attention has been focused on reducing the high prevalence of clinician
burnout in the U.S. The National Academy of Medicine (2018) launched an Action Collaborative
to advance research and harness interdisciplinary initiatives to improve clinician well-being.
Additionally, findings from the 2018 National Physician Burnout and Depression survey showed
physicians in primary care to have one of the highest rates, 47%, of burnout (Peckham, 2018).
Despite the growing attention given to clinician burnout in the U.S., there remains a knowledge
gap on the overall prevalence, predictors, and outcomes of burnout among all primary care
providers (PCPs) that includes physicians, nurse practitioners (NPs), and physician assistants
(PAs) (U.S. Department of Health and Human Services, 2016).
17
Primary care improves population health (van Weel & Kidd, 2018) and serves as the first
point of entry to the health care system. Greater access to primary care is associated with
improved health outcomes and decreased utilization of costly acute care services such as
hospitalizations and emergency department use (Shi, 2012; Chang et al., 2011; Kravet et al.,
2008). In the U.S., 72.6% of all NPs (AANP, 2019), 26.7% of certified PAs (NCCPA, 2019), and
28% of physicians actively deliver primary care to patients (Dall et al., 2018), yet minimal
evidence exists on burnout inclusive of all PCPs, which is concerning since primary health care
is delivered by an interdisciplinary team of providers (Phillips & Bazemore, 2010). Most
research to date has been focused on exploring predictors and outcomes of burnout among
physicians in many clinical specialties (Panagioti et al., 2018; Wen et al., 2016; Tawfik et al.,
2018), demonstrating that physician burnout is associated with poor quality of patient care,
compromised patient safety (Panagioti et al., 2018; West et al., 2018), and medical errors (Wen
et al., 2016). As NPs and PAs are unique types of PCPs who may be at risk for burnout, it is
important to investigate the predictors and outcomes of burnout inclusive of all PCPs to ensure
delivery of safe, high quality patient care.
Researchers have found that predictors of clinician burnout are often modifiable and
include the practice environment, payment models, and policies governing professional practice,
licensure, and reimbursements (Brigham et al., 2018). In a systematic review and meta-analysis
of 47 studies, burnout was associated with compromised patient safety, low clinician
professionalism, low quality of care, and low patient satisfaction (Panagioti et al., 2018).
Although several factors can influence burnout and its subsequent outcomes, there has yet to be a
comprehensive review of literature on the relationship between provider burnout, predictors, and
outcomes with a specific focus on primary care. With primary care having a significant role in
18
the health care system (Shi, 2012) and PCPs being at the forefront (Corley, 2017), it is
imperative to understand the factors that increase the risk of provider burnout and the subsequent
consequences in primary care. Therefore, the purpose of this systematic review was to identify
the predictors and outcomes of burnout among PCPs within the U.S.
2.11 New Contributions
This systematic review comprehensively investigates the predictors and outcomes of
burnout among PCPs in the U.S. and makes new contributions to the existing evidence.
Researchers have previously studied burnout among physicians (Panagioti et al., 2018) and PAs
and NPs (Hoff, Carabetta, & Collinson, 2017) in diverse health care settings and specialties not
limited to primary care. However, it is important to investigate the predictors and outcomes of
provider burnout specifically in the primary care setting because most health care in the U.S. is
delivered in primary care practices (Starfield, Shi, & Macinko, 2005). Thus, evidence focused on
hospital-based clinicians cannot be generalized to PCPs. In addition, prior studies and reviews
are not inclusive of all PCPs (i.e., physicians, NPs, and PAs), which is a limitation since primary
care in the U.S. is often delivered by an interdisciplinary team of providers including NPs, PAs,
and physicians. Therefore, examining burnout among all types of PCPs will allow for the
development of future interventions to reduce PCP burnout.
2.12 Conceptual Framework
This review is guided by the Clinician Well-Being and Resilience model which depicts
external and internal predictors associated with clinician well-being while illustrating the
relationship between clinician well-being with clinician, patient, and organizational outcomes
(Brigham et al., 2018). According to the model, external predictors that can influence clinician
well-being and burnout include: 1) practice environment, 2) organizational factors (e.g.,
19
harassment, workload), 3) health care responsibilities, 4) rules and regulations (e.g., national and
state policies governing clinical practice, documentation requirements), and 5) society/culture
(e.g., stigmatization of burnout) (National Academy of Sciences, 2018). The internal predictors
within the model are 1) personal factors (e.g., work-life balance, sense of meaning, inclusion and
connectivity, personal values) and 2) skills and abilities (e.g., clinical competency, coping,
communication) (National Academy of Sciences, 2018).
The Clinician Well-Being and Resilience model guided our review by allowing us to
categorize the predictors (i.e., personal and organizational) and outcomes (i.e., patient, provider,
and organizational) of PCP burnout. Furthermore, the model suggests that regulatory factors can
influence clinician well-being. This implies that scope of practice regulations, which are
common for primary care NPs and PAs, may impact clinician well-being. Thus, this model
enabled a conceptually-motivated process in examining the current evidence on burnout among
primary care physicians, NPs, and PAs.
2.13 Methods
Data Sources and Searches: The Preferred Reporting Items for Systematic Reviews and
Meta-analyses (PRISMA) guided this review (Moher et al., 2009). We published a study
protocol of this review in Prospero, an international database of prospective systematic reviews
(Abraham, Zheng, & Poghosyan, 2018). We consulted with a library informationist who
provided guidance in the development of our search strategy. Two researchers completed a
comprehensive literature search in eight electronic databases: PubMed, Embase, Cumulative
Index to Nursing and Allied Health Literature (CINAHL), Cochrane Library, Ovid Medline,
ProQuest, Joanna Briggs Institute, and PsychINFO. ProQuest was searched for gray literature.
The search strategy consisted of medical subject headings (MeSH) as well as database specific
20
subject headings to identify studies that reported predictors and/or outcomes of burnout among
PCPs in the U.S. Keywords included: “burnout,” “burn* out,” “nurse practitioner*,” “advance*
practice nurse*,” “physician assistant*,” “physician*,” “clinician*,” “doctor*,” “primary care
provider*,” “primary care,” “predictor*,” “factor*,” “cause*,” “outcome*,” “associate*,”
“lead*,” “led*,” and “consequence*” (Appendix A). Domestic studies were identified using the
Boolean term “AND” with burnout, nurse practitioners or advanced practice nurses, physicians,
physician assistants, clinicians, doctors, and primary care. The search was not limited by a set
timeframe as we wanted to include all relevant studies discussing predictors and/or outcomes of
PCP burnout. Manual searches of reference lists from the retrieved articles were conducted but
no new studies were identified. The literature search was first conducted from 01/01/2019 to
02/03/2019 and an updated search was conducted from 9/30/2019 to 10/02/2019 to ensure that
all studies that met inclusion criteria were included.
Study Selection and Eligibility for Full-text Review: The PRISMA flow diagram of the
literature search is depicted in Figure 1. The literature search yielded 2,313 studies, all of which
were exported into EndnoteX8 citation management software. After removal of duplicates, 1,772
studies remained and underwent initial screening. Of these, 68 studies were deemed relevant by
title and abstract screening and were then reviewed by full-text using the following eligibility
criteria. Studies were included if they: 1) were empirical research studies, 2) focused on
predictors and/or outcomes of burnout among PAs, NPs, or physicians exclusively, or in
combination, 3) took place in any primary care setting in the U.S., and 4) were written in
English. Only studies conducted in the U.S. were included because the PCP workforce may
differ internationally since not all countries have an NP and/or PA primary care workforce.
Studies were excluded for the following reasons: 1) study samples were limited to students
21
instead of practicing PCPs; 2) studies were conducted in acute care settings; 3) studies were
conducted outside the U.S.; 4) studies did not focus on predictors or outcomes of burnout in
PCPs; 5) studies were unrelated to burnout in PCPs; 6) studies were editorials, news/policy
briefs, or commentaries; and 7) studies evaluated programs instead of investigating predictors or
outcomes of PCP burnout. Of the 68 studies reviewed in full-text, 47 were excluded because
researchers did not focus on providers practicing in primary care (n = 10 studies), study authors
did not discuss predictors or outcomes of burnout in PCPs (n = 31), studies occurred in an
international setting (n = 2), study authors addressed burnout in health professional students (n =
2), and study authors evaluated programs instead of comprehensively investigating predictors or
outcomes of PCP burnout (n = 2), resulting in a total of 21 articles eligible for inclusion in this
review.
Assessment of Methodological Quality and Risk of Bias: Two authors independently
appraised the quality of included studies using the Joanna Briggs Institute (JBI) critical appraisal
checklists for cross-sectional and cohort studies (Moola et al., 2017). Both JBI checklists were
developed to help investigators determine the risk of bias and methodological quality of studies.
The checklists include questions assessing clarity of inclusion criteria, description of participants
and recruitment, measures of variables, use of confounding variables, statistical analyses, and
methods to address attrition (Moola et al., 2017). Response options for each question include
“yes,” “no,” “unclear,” or “not applicable.” Consistent with previous researchers who have used
the JBI checklist (de Cordova et al., 2018), studies had to receive a “yes” for at-least five out of
the eight questions (62.5%) in the cross-sectional study checklist and a “yes” for at-least seven
out of the eleven questions (63.6%) in the cohort study checklist to be considered sufficient in
22
quality for inclusion in this review. The more “yes” ratings a study has, the higher the quality
and lower the risk of bias.
Data Extraction: The study purpose, setting, sample size, PCP type, response rate,
instrument used to measure PCP burnout, predictors and/or outcomes that were significantly
associated with PCP burnout, and the prevalence of burnout from each study are presented in
Table 1. Two supplementary tables, summarizing the predictors and outcomes across studies,
were created to concisely display the results within this review. To reduce the potential for bias,
two authors independently extracted information from each study and then compared results. If
there was a lack of clarity about the study, we contacted the first author of the study for further
clarification.
2.14 Results
Study Characteristics: Of the included 21 studies, 19 studies used cross-sectional
designs and two studies used cohort designs. The studies were conducted in several U.S. states
with some study authors using a national sample of PCPs (Simonetti et al., 2017; Edwards et al.,
2017; Spinelli et al., 2016; Helfrich et al., 2014; Yoon et al., 2017; Zubatsky et al., 2018;
Helfrich et al., 2017). Across studies, the sample size ranged from 96 to 4,130 PCPs, and most
studies (n = 13) used a single item (Rohland, Kruse, & Rohrer, 2004; Dolan et al., 2015) to
measure burnout. Overall, prevalence of PCP burnout ranged from 13.5% to 60%. Predictors of
PCP burnout consisted of 1) personal and 2) organizational factors. Outcomes of PCP burnout
occurred at the 1) patient, 2) provider, and 3) organizational level.
Quality Appraisal: All 21 studies were deemed of sufficient quality and were included
in this review. Cross-sectional studies had appraisal scores between six to all eight questions
with a “yes” rating (Table 2), and cohort studies had seven to all 11 questions receiving a “yes”
23
rating (Table 3). Fifteen cross-sectional studies received all “yes” ratings in quality appraisal.
Three studies did not have clear information on either the confounding factors included in their
analysis (Blechter et al., 2017; Whitebird et al., 2017) or the use of all valid and reliable outcome
measures (Robertson et al., 2017). One study did not provide a detailed overview of study
subjects and setting (Rabatin et al., 2016), whereas one cohort study received a seven out of 11
because it was unclear whether 1) predictors were measured similarly among PCPs who were
burned-out versus not burned-out, 2) participants were free of the outcome at the start of the
study, and 3) reasons for loss to follow-up were not sufficiently described (Ratanawongs et al.,
2008).
Predictors of PCP Burnout: Several study authors demonstrated that PCP
characteristics were predictors of burnout, including length of tenure at current practice, sex,
personal values, a “sense of calling” to medicine, career satisfaction, satisfaction with resources
needed for patient care, and perceptions of being able to care for complex patients.
Organizational predictors were modifiable characteristics of the primary care practice where
PCPs work and included practice environment, working status, teamwork, electronic health
records, and the size and setting of the health care system.
Personal Predictors. Length of tenure at the current primary care practice was associated
with burnout, with mixed evidence existing on whether longer or shorter tenure increases PCP
burnout (Olayiwola et al., 2018; Helfrich et al., 2014; Edwards et al., 2018; Whitebird et al.,
2017; Zubatsky et al., 2018). In a study with 359 PCPs from San Francisco, PCPs working in the
practice for more than five years had lower emotional exhaustion scores (p < .05) compared to
PCPs working less than one year (Olayiwola et al., 2018). Similarly, PCPs working more than 10
years reported lower emotional exhaustion than those working less than six years (Zubatsky et
24
al., 2018). However, PCPs with two or more years of tenure had higher odds of burnout (odds
ratio (OR) = 2.13 to 2.68) compared to PCPs with less than six months of tenure (Helfrich et al.,
2014). Consistent with these findings, a different group of researchers reported that PCPs with
greater tenure in medical practice had higher burnout (Whitebird et al., 2017), and another found
that PCPs with greater than three years of employment at the current practice had a 1.48 higher
odds of burnout (95% confidence interval (CI) = 1.32 to 1.64; p < .001) compared to PCPs with
three or less years of tenure at the current practice (Edwards et al., 2018).
The findings were mixed regarding sex as a predictor of PCP burnout. One study found
that female physicians had higher levels of burnout compared to male physicians (36.0% versus
19.0%, p < .001) (Rabatin et al., 2016). Similarly, another study found lower levels of emotional
exhaustion in male physicians compared to female physicians (Zubatsky et al., 2018). However,
in a different study, burnout was higher among males compared to females (38.6% versus 27.4%,
p = .117) (Spinelli et al., 2016).
Additionally, primary care physicians were less likely to be burned-out and were more
likely to stay in their current practice if their values aligned with that of administration and
leadership (p < .05) (Babbot et al., 2014). Furthermore, a “sense of calling” implies that PCPs
have an inner purpose, are motivated, and view their profession as a central aspect of their lives
(Yoon et al., 2017). Using a national survey of 896 primary care physicians, physicians with a
high “sense of calling” into medicine were less likely to report burnout compared to those with a
low sense of calling (OR = 0.4; 95% CI = 0.3 to 0.7; p = .002) (Yoon et al., 2017). In a separate
study with PCPs practicing in 172 clinics across eight states, researchers found that lower PCP
career satisfaction (p < .001), lower satisfaction with resources needed to care for complex
25
patients (p < .001), and lower perceptions of being able to care for complex patients were
associated with greater symptoms of burnout (p < .001) (Whitebird et al., 2017).
Organizational Predictors. Various features of the practice environment were the most
frequently reported predictors of PCP burnout. For example, in a study with 422 primary care
physicians, burnout increased when work control, emphasis on quality, communication,
organizational trust, and workplace cohesiveness all decreased (p < .05). Whereas, in a different
cross-sectional study involving 204 physicians and 31 NPs/PAs from 174 primary care practices
in New York City, the odds of PCP burnout were lower (OR = 0.12; 95% CI = 0.002 to 0.85) in
practices with strong leadership, communication between members, trust, and teamwork
(Blechter et al., 2017).
Poor work control and greater time pressure were commonly reported factors
contributing to primary care physician burnout (p < .05) (Rabatin et al., 2016; Linzer et al., 2009;
Babbot et al., 2014). Additionally, burned-out physicians were four times more likely to be in
chaotic workplaces (p < .001) and were four times less likely to be in an environment that
promoted work-life balance (p < .001) (Rabatin et al., 2016). Similarly, chaotic practice
environments that made PCPs feel overwhelmed with the demands of their job were associated
with burnout (OR= 4.33; 95% CI = 3.78 to 4.96) (Helfrich et al., 2014), as well as high
workloads as measured by having patient panels beyond the recommended capacity (OR = 1.19,
95% CI = 1.01 – 1.40) (Helfrich et al., 2017). Staff turnover was associated with increased PCP
burnout (Edwards et al., 2017; Helfrich et al., 2017), yet adequately staffed primary care
practices were associated with reduced burnout (Edwards et al., 2017; Helfrich et al., 2014;
Helfrich et al., 2017).
26
A PCP’s work schedule (i.e., full-time status and hours worked per week) also predicted
burnout. For example, PCPs employed full-time had higher burnout compared to those who
worked part time (33.9% versus 24.6%, p = .03) (Spinelli et al., 2016). Similarly, PCPs working
greater than 40 hours per week were 1.88 times more likely to be burned-out (95% CI = 1.64 to
2.15; p < .001) compared to PCPs working 40 hours or less per week (Edwards et al., 2018).
Furthermore, burnout was associated with working extended hours during the weekend (OR=
1.48; 95% CI = 1.06 to 2.05) (Helfrich et al., 2017).
In addition, a study involving 231 PCPs and 280 staff members across 16 primary care
clinics in San Francisco, found that when PCPs were working in teams with greater structure
which was defined as the PCP consistently working with the same staff (p < .05), and greater
organizational culture which was defined as the perception that PCPs had of working effectively
with their colleagues (p < .001), PCPs felt less emotionally exhausted (Willard-Grace et al.,
2013). The benefits of team-based primary care were also seen in a national study with 288
primary care physicians who reported lower burnout (e.g., depersonalization [β = -2.48, 95% CI
= -4.54 to -0.42]) with higher levels of integrated behavioral health services within primary care
(Zubatsky et al., 2018). When PCPs were not working in teams, certain primary care tasks were
associated with burnout. For example, providing patient education on self-care activities (β =
3.83, p = .03, 95% CI = 0.33 to 7.32) and lifestyle changes (β = 4.11, p = .03, 95% CI = 0.39 to
7.83) contributed to PCP burnout (Kim et al., 2018). However, effective team communication
reduced PCP burnout (p < .001) (Kim et al., 2018). In addition, among 2,481 PCPs, the
prevalence of burnout was 2.5% higher in physicians than in NPs/PAs [Physicians = 25.1%
burned-out; NPs/PAs = 22.6% burned-out] (Edwards et al., 2018).
27
Furthermore, providers employed in primary care practices transitioning to electronic
health records (EHRs) reported higher emotional exhaustion (β = 0.80; 95% CI = 0.16 to 1.44; p
< .05) compared to PCPs in practices that were not in EHR transition (Willard-Grace et al.,
2013). Seventy-five percent of primary care physicians and resident physicians attributed their
feelings of burnout to greater use of complex EHRs; particularly, those physicians and resident
physicians who spent more than six hours per week using the EHR outside of their assigned
work hours were 2.9 times more likely to report burnout (95% CI = 1.9 to 4.4) compared to those
who spent six hours or less on the EHR (Robertson et al., 2017). In addition, primary care
physicians in practices with moderate use of multifunctional EHRs had higher burnout compared
to physicians in practices that used minimal features of the EHR (p = .08) (Babbott et al., 2014).
Primary health care system characteristics such as the size and setting also contribute to
PCP burnout, with the odds of burnout higher among PCPs working in non-solo practices
compared to solo practices (OR = 1.71, 95% CI = 1.35 to 2.16) (Edwards et al., 2018). Similarly,
PCPs in a hospital/health system owned practice (OR = 1.42, 95% CI = 1.16 to 1.73) and
federally qualified health centers (OR = 1.36, 95% CI = 1.03 to 1.78) had higher odds of burnout
than PCPs in physician owned practices (Edwards et al., 2018). Furthermore, limited capacity
and lack of organizational resources to address the needs of patients increased the risk of PCP
burnout, yet, greater clinic resources were associated with lower emotional exhaustion in PCPs
(β = -0.41, 95% CI = -0.82 to -0.01, p < .05) (Olayiwola et al., 2018).
Outcomes of PCP Burnout: Patient Level. Most studies (n = 15) did not investigate any
outcomes of PCP burnout. Of the six studies that reported outcomes, there were mixed findings
between PCP burnout and patient outcomes. Specifically, two studies reported no associations
between PCP burnout and quality of patient care and medical errors (Rabatin et al., 2016; Linzer
28
et al., 2009), blood pressure and diabetes care management, screening for colon cancer,
depression, or use of tobacco (Rabatin et al., 2016). However, others suggested PCP burnout was
associated with negative rapport-building statements (i.e., criticism) during a provider-patient
interaction (p < .001) (Ratanawongs et al., 2008) and an increase in the likelihood of medical
errors (p < .05) and poor patient care (p < .01) (Williams et al., 2007). Additionally, physician
burnout was neither associated with differences in patient satisfaction (Ratanawongs et al.,
2008), nor antibiotic prescribing for patients with acute respiratory infections in primary care
(Sun et al., 2017).
Provider Level. One multi-site study investigated the association between physician
burnout and well-being (Rabatin et al., 2016). In particular, lower job satisfaction and job-
associated stress were reported outcomes of primary care physician burnout. Specifically,
burned-out physicians were less likely to be satisfied with their present job, since only 9% of
burned-out physicians reported being satisfied with their job compared to 59% of non-burned out
physicians who were satisfied (p < .001) (Rabatin et al., 2016). In addition, physicians
experiencing burnout were more likely report high job stress (p < .001) (Rabatin et al., 2016).
Organizational Level. Burnout is associated with poor organizational outcomes such as
PCPs leaving the practice. In a longitudinal cohort study, nearly 53% of PCPs reported burnout;
and job turnover was more common among burned-out PCPs (OR = 1.57, 95% CI = 1.02 to 2.40,
p = .04) (Willard-Grace et al., 2019). In a different study, PCP burnout was associated with PCPs
leaving their organization (21% non-burned-out versus 56% burned-out, p < .001) (Rabatin et al.,
2016).
29
2.15 Discussion
To our knowledge, this is the first systematic review to explore predictors and outcomes
of burnout among all types of providers practicing in primary care practices in the U.S. Overall,
there was a wide range in the prevalence of PCP burnout, with 60% being the highest reported
rate. Studies reported either personal predictors or modifiable organizational predictors
contributing to PCP burnout, with few exploring outcomes.
The findings of the reviewed studies were mixed regarding the association of sex and
length of tenure on PCP burnout. The mixed findings in whether male or female clinicians or
those with longer or shorter tenure report burnout was consistent with existing literature which
reports that one particular sex does not have a greater risk for burnout (Purvanova & Muros,
2010). Additionally, the reviewed studies used different instruments to measure burnout and had
an unequal number of males and females (Spinelli et al., 2016; Rabatin et al., 2016) which may
potentially explain the mixed findings regarding sex as a predictor of burnout. For example,
Spinelli and colleagues (2016) measured burnout using the Maslach Burnout Inventory (MBI). In
addition, they had 423 female participants and 44 male participants, of which 96 were PCPs yet
the prevalence of PCP burnout by sex was not reported (Spinelli et al., 2016). Whereas, Rabatin
and colleagues (2016) measured burnout using a single item, and had 187 female and 235 male
participants included in their study. Similarly, mixed findings regarding length of tenure and
PCP burnout may be explained by the differences in practice characteristics. For example,
Helfrich and colleagues (2014) used a national sample of PCPs from the Veteran’s Health
Administration (VA), and these PCPs may differ from PCPs not affiliated with the VA. In
contrast, Olayiwola and colleagues (2018) investigated burnout specifically among PCPs from
three delivery systems in San Francisco. Since these PCPs are not a representative sample of all
30
PCPs in the U.S, mixed findings regarding length of tenure and burnout are likely to occur.
Future research using consistent measurement tools, a national sample of PCPs with similar
representation of males, females, and PCPs with varying levels of clinical experience will help to
address some of the limitations seen in existing studies.
Most studies report organizational predictors of burnout, suggesting that characteristics of
primary care practices can lead to PCP burnout. Our findings are consistent with other
researchers who found that a poor practice environment is a predictor of burnout among
registered acute care nurses (Van Bogaert et al., 2013). Since much of the organizational
predictors of PCP burnout appear to be modifiable, this gives clinicians and administrators an
opportunity to intervene and improve the primary care practice environment to ultimately reduce
PCP burnout. Based on our results, improving the practice environment may be done by ensuring
that PCPs have adequate staff and resources for patient care, have autonomy over their practice,
and have technical assistance readily available when using the EHR.
While several studies in this review examined factors associated with PCP burnout, only
six out of 21 studies investigated the consequential outcomes. Although there was a limited
number of outcome-related studies on PCP burnout, other studies on clinician burnout have
found burnout to be adversely associated with patient care, provider well-being, and
organizational outcomes (West et al., 2018). Researchers investigating outcomes of burnout
among nurses (Liu & Aungsuroch, 2017) and surgeons (Shanafelt et al., 2010) found that
burnout was associated with lower quality of patient care, and such findings were comparable in
a systematic review with international studies in which physician burnout was associated with
poorer patient care (Panagioti et al., 2018). In contrast to Panagioti and colleagues (2018), our
review showed mixed results on the association between PCP burnout and quality of care in
31
primary care practices in the U.S., which may stem from differences in methodology since we
included neither international studies nor studies conducted in the acute care setting. Thus, we
cannot conclude that PCP burnout is adversely associated with lower quality of care, but our
review does suggest that burnout influences PCPs (i.e., job satisfaction). In particular, lower job
satisfaction among PCPs experiencing burnout may increase PCP turnover, which can
subsequently compromise patient care due to the limited availability of PCPs. The relationship
between turnover and PCP burnout should be explored in future studies.
Furthermore, all of the studies in this review investigated burnout among primary care
physicians. Of the studies that included other PCPs such as NPs and PAs, these researchers
grouped NPs and PAs into one category (Edwards et al., 2018; Olayiwola et al., 2018; Willard-
Grace et al., 2019; Willard-Grace et al., 2013). Categorizing NPs and PAs as a single group is
problematic because NPs and PAs come from two different professions with distinct licensures
and regulations. As a result, the disaggregated rates of burnout across NPs and PAs remain
unclear. Furthermore, only one study in this review (Willard-Grace et al., 2019) included NPs
and PAs when investigating outcomes associated with PCP burnout. The scarcity of burnout
research focused on PCPs such as NPs and PAs is a gap in the literature and problematic because
NPs and PAs are increasingly delivering primary care to patients (Auerbach, Straiger, &
Buerhaus, 2018).
Our systematic review provides novel findings compared to previously conducted
reviews. For example, in a systematic review and meta-analysis on the prevalence of burnout in
primary care registered nurses, who have different scope of practice regulations than NPs, nearly
28% of nurses reported high emotional exhaustion- a component of burnout (Monsalve-Reyes et
al., 2018). However, Monsalve-Reyes and colleagues (2018) investigated neither predictors nor
32
outcomes associated with nurse burnout, yet they state that nurse practice environments (e.g.,
increased workload and lack of control over their work) may increase feelings of burnout, which
is consistent with the findings from our review. Furthermore, previous reviews on physician
burnout included international studies with physicians from specialties not limited to primary
care (e.g., surgery, palliative care, neurology, emergency medicine, anesthesia, urology)
(Rotenstein et al., 2018; Panagioti et al., 2018). As a result, our findings add new evidence as we
specifically focused on primary care and studies conducted in the U.S.
Strengthens and Weaknesses of Study Designs: The studies included in this review had
some methodological strengths. Several studies had a large representative sample of PCPs
(Edwards et al., 2018; Yoon et al., 2017) practicing in different primary care settings in rural and
urban locations as well as in VA clinics (Helfrich et al., 2014; Simonetti et al., 2017; Edwards et
al., 2017; Helfrich et al., 2017). Some studies also had survey response rates greater than 80%
(Willard-Grace et al., 2019; Williams et al., 2007; Edwards et al., 2018; Zubatsky et al., 2018),
which reduces the risk for non-response bias.
Despite these strengths, the studies in our review had methodological challenges similar
to studies in other reviews on clinician burnout (Hoff et al., 2017; Rotenstein et al., 2018). Most
studies in our review did not have longitudinal designs which limited our ability to determine
whether certain predictors cause burnout or burnout causes specific outcomes. Across studies,
there was a lack of consistency in how burnout was defined and measured which can create
variability in the results. Some authors defined PCP burnout only by the presence of emotional
exhaustion (Willard-Grace et al., 2013; Kim et al., 2018), some included symptoms of emotional
exhaustion, depersonalization, and feelings of low personal accomplishment within their
evaluation of PCP burnout (Spinelli et al., 2016; Sun et al., 2017; Zubatsky et al., 2018), some
33
included only two of the three symptoms (Ratanawongsa et al., 2008), and two study authors
used some or all subscales (i.e., exhaustion, cynicism, and professional efficacy) within the MBI-
general survey (Olayiwola et al., 2018; Willard-Grace et al., 2019). The remaining study authors
used a single survey item which asked PCPs to use their own definition of burnout to rate their
current level of burnout.
In addition, several cross-sectional studies did not control for a comprehensive set of
variables which may bias the overall results (Rabatin et al., 2016; Spinelli et al., 2016; Robertson
et al., 2017). For example, some authors included work hours as a covariate in the analysis but
did not control for PCP workload which is concerning since high workloads can confound the
relationship between burnout and potential outcomes (Babbott et al., 2014; Blechter al., 2017).
Additionally, some study authors (Robertson et al., 2017; Babbott et al., 2014) did not include
important EHR covariates (e.g., user experience and training with the EHR, EHR type and
complexity, organizational support with the EHR) (Yen & Bakken, 2012) in their analyses which
is concerning because omitting them can affect the results. Furthermore, a major limitation
associated with cross-sectional study designs is the presence of unmeasured or unobservable
confounders, which may be influencing the relationship between predictors, outcomes, and
burnout, and thus lead to spurious results. For example, it may be possible that the unmeasured
variable of low self-efficacy may influence both feelings of career dissatisfaction (a predictor of
PCP burnout) and burnout. As a result, low self-efficacy can make the relationship between
career dissatisfaction and burnout specious. Lastly, several studies were conducted in a single
city (e.g., San Francisco, New York City, Baltimore) which limits the generalizability of findings
to PCPs living in other parts of the U.S. (Willard-Grace et al., 2019; Ratanawongs et al., 2008;
Olayiwola et al., 2018; Blechter et al., 2017). Nonetheless, our review and prior reviews suggests
34
the importance of conducting future burnout studies with robust study designs (e.g.,
longitudinal), large representative samples of PCPs, and the use of consistent tools to measure
and define burnout.
2.16 Implications for Clinical Practice and Research
The findings of this review have implications for practice and research. To reduce PCP
burnout, at the practice level, clinicians and administrators can take actions to improve the
primary care practice environment. Our findings suggest that improving the practice environment
may be done by fostering team culture, increasing PCP autonomy, promoting work-life balance,
and ensuring a sufficient supply of staff and resources needed to provide patient care. This
review also highlights the importance of administrators providing support to PCPs during the
implementation of EHRs, since use of EHRs may be associated with PCP burnout.
Furthermore, some studies reported that not working in teams (Kim et al., 2018), staff
turnover, and high workloads (e.g., having patient panels beyond reasonable capacity) were
associated with PCP burnout (Helfrich et al., 2017; Edwards et al., 2017), yet greater
collaboration (i.e., integration between behavioral health and primary care services) reduces
burnout (Zubatsky et al., 2018). These findings suggest that collaboration within the primary care
setting may allow PCPs to distribute their workload to potentially reduce feelings of burnout.
Moreover, there remains a scarcity of research investigating burnout among primary care NPs
and PAs. Additional research is needed to identify the rates of burnout, as well as the predictors
and outcomes associated with primary care NP and PA burnout. Supported by our conceptual
model, it is important to include primary care NPs and PAs in burnout research because their
experiences of burnout may be influenced by rules and regulations governing their ability to
practice. However, none of the studies included in this review examined the influential role that
35
regulations have on PCP burnout, yet in an international study, rules and regulations were
recognized as a source of burnout for pediatric hospital nurses (Bilal & Ahmed, 2017).
More research is needed to identify outcomes associated with PCP burnout in the U.S.
Internationally, burnout is associated with the abuse of alcohol and drugs in PCPs (Soler et al.,
2008), fatigue and irritability (Abdulla, Al-Qahtani, & Al-Kuwari, 2011), and lower patient
satisfaction (Anagnostopoulos et al., 2012). The limited outcomes-related research in the U.S.
indicates a need for more qualitative and quantitative research with robust study designs (e.g.,
longitudinal studies). Using these study designs, we can further investigate how PCP burnout is
associated with patient safety, quality of care, provider well-being, and organizational outcomes.
2.17 Limitations
Although a comprehensive search was conducted, this review focused on studies
published in English, creating the potential for missing relevant studies in other languages. This
study is also limited to PCPs in the U.S., thereby excluding relevant studies in other countries.
However, primary care delivery may differ in other countries thereby increasing the risk for
heterogeneity in our findings if international studies were included.
2.18 Conclusion
Across the U.S., physicians, NPs, and PAs, are delivering primary care services. The
demand for their services is expected to grow given an aging population with chronic conditions.
Without support from administrators, PCPs experiencing high workloads, multiple job demands,
and limited staffing may be susceptible to burnout. Intervening to improve the primary care
practice environment may hold potential for reducing the high rates of burnout among PCPs in
the U.S. Despite the timeliness and importance of addressing PCP burnout, current studies have
various design flaws, and so future studies with robust study designs, clear definitions and
36
standardized instruments to measure burnout are needed to advance the knowledge on PCP
burnout.
37
2.19 Figure 1
PRISMA Flow Diagram Illustrating the Literature Search
38
2.20 Table 1
Characteristics of Included Studies (n = 21)
Author, Year Study Purpose
PCP Sample, Setting, Response Rate, Study Design
Measure of Burnout
Predictors Outcome & Burnout Prevalence in PCPs
Willard-Grace et al., 2013
Determine relationship between emotional exhaustion in primary care workers with team structure and culture
PCP sample • Physicians
(n = 115) • NP/PAs
(n = 31) • Residents
(n = 85) Setting = • San
Francisco, California
Response Rate = • 55% Study Design = • Cross-
sectional
MBI Organizational = • Team
structure • Team
culture • Transition
to EHR • Job role
(e.g., being a resident)
Personal = • Longer
tenure • More half
days worked
Outcomes = • NR Prevalence = • 60.0%
high emotional exhaustion
Rabatin et al., 2016
Investigate relationship between work conditions, burnout, quality of care, and medical errors in primary care
PCP sample = • Family
physicians and general internists (n = 422)
Setting = • New York
City, New York; Chicago, Illinois; Milwaukee, Madison, and smaller towns in Wisconsin
Response Rate = • 56%
Rohland et al., (2004) single item burnout measure
Organizational = • Poor work
control • Greater
time pressure
• Work-life imbalance
• Chaotic work environment
Personal = • Gender
Organizational = • Intention
to leave practice
Provider = • Lower
physician job satisfaction
• Greater job stress
Patient = • No
significant association between burnout &
39
Study Design = • Cross-
sectional
patient outcomes
Prevalence = • 36.0% in
female physicians
• 19.0% in male physicians
Simonetti et al., 2017
Determine whether patient-centered medical home implementation is related to PCP burnout and estimate the variation in the prevalence of burnout from 2012 to 2013
PCP sample = • Physicians,
NPs, & PAs (n = 4130)
Setting = • Veterans
Affairs primary care clinics in the U.S.
Response Rate = • 62.3% in
2012 and 56.3% in 2013
Study Design = • Cross-
sectional
Rohland et al., (2004) single item burnout measure
Organizational = • Job role at
practice
Outcomes = • NR Prevalence = • 45.5% in
PCPs
Edwards et al., 2017
Determine whether task delegation is associated with PCP burnout
PCP sample = • Physicians,
NPs, & PAs (n = 721)
Setting = • Veterans
Affairs primary care clinics in the U.S.
Estimated Response Rate = • 21% Study Design = • Cross-
sectional
Rohland et al., (2004) single item burnout measure
Organizational = • Workload
(i.e., task delegation & reliance)
• Task discordance
• Staffing • Staff
turnover • Presence of
a PACT coach
Personal = • Years of
VA
Outcomes = • NR Prevalence = • 48.0% in
PCPs
40
employment
Blechter et al., 2017
Determine prevalence of burnout and factors associated with burnout in PCPs in small primary care practices
PCP sample • Physicians
(n = 204) • NPs/PAs
(n = 31) Setting = • New York
City, New York
Response Rate = • NR Study Design = • Cross-
sectional
Rohland et al., (2004) single item burnout measure
Organizational = • Adaptive
reserve score*
Outcomes = • NR Prevalence = • 13.5% in
PCPs
Spinelli et al., 2016
To use the MBI and the Areas of Worklife Survey to identify the prevalence and workplace factors associated with burnout in PCPs and staff
PCP sample = • Physicians,
advanced practitioners, & mental health providers (n = 96)
Setting = • Midwestern
region of the U.S., specific city/state NR
Response Rate = • 55.7% Study Design = • Cross-
sectional
MBI Organizational = • Full-time
work status • Job role • ^ Work
environment
Personal = • Gender • Tenure
Outcomes = • NR Prevalence = • 37.5% in
PCPs
Helfrich et al., 2014
Determine whether team-based primary care is related to lower burnout in PCPs in the
PCP sample = • Physicians,
NPs, & PAs (n = 1769)
Setting = • Veterans
Affairs primary care
Rohland et al., (2004) single item burnout measure
Organizational = • Job role • Being
assigned to a PACT
• Staffing
Outcomes = • NR Prevalence = • 45.0% in
PCPs
41
Veteran Affairs PACT
clinics in the U.S.
Response Rate = • NR Study Design = • Cross-
sectional
• Chaotic work environment
• Work demands
• Participatory decision making
Personal = • Longer
tenure (2 or more years)
Olayiwola et al., 2018
Investigate relationship between PCP burnout and clinic capacity to address the social needs of patients
PCP sample • Physicians
(n = 174) • Resident
physicians (n = 141)
• NPs/PAs (n = 44)
Setting = • San
Francisco, California
Response Rate = • 71% Study Design = • Cross-
sectional
MBI-General Survey
Organizational = • Job role • Organizatio
nal capacity to address patients’ social needs
Personal = • Tenure • Half days
worked
Outcomes = • NR Prevalence = • 51.2% of
PCPs had high exhaustion
• 40.1% had high cynicism
• 23.0% had feelings of low professional efficacy
Linzer et al., 2009
Determine relationship between poor primary care work conditions with patient care, physician stress, burnout, and intention to leave practice
PCP sample • Family
Physicians (n = 204)
• General Internists (n = 218)
Setting = • New York
City, New York; Chicago, Illinois; Milwaukee, Madison,
Rohland et al., (2004) single item burnout measure
Organizational = • Time
pressure during patient care visits
• Chaotic work environment
• Work control
• Organizational culture
Patient = • No
association between burnout and patient care (quality and medical errors)
Prevalence = • 26.5% in
primary
42
and smaller towns in Wisconsin
Response Rate = • 56% Study Design = • Cross-
sectional
care physicians
Robertson et al., 2017
Determine prevalence of burnout, satisfaction with work-life balance, and the relationship between practice and primary care physicians’ and resident physicians’ characteristics with burnout
PCP sample • Physicians
(n = 245) • Resident
physicians (n = 340)
Setting = • Virginia,
North Carolina, South Carolina, and Florida
Response Rate = • 68% Study Design = • Cross-
sectional
Rohland et al., (2004) single item burnout measure
Organizational = • EHR • Greater
out-of-work hours spent using the EHR
Personal = • Gender
Outcomes = • NR Prevalence = • 37.0% in
primary care physicians
Babbott et al., 2014
Examine relationship between EHR functions, work conditions with physician burnout, stress, and satisfaction
PCP sample = • At baseline,
family physicians and general internists (n = 422)
Setting = • New York
City, New York; Chicago, Illinois; Milwaukee, Madison, and smaller towns in Wisconsin
Rohland et al., (2004) single item burnout measure
Organizational = • Use of
multifunctional EHRs
• Time pressure during patient care visits
• Work control
• Lack of organizational emphasis on quality, communica
Outcomes = • NR Prevalence = • NR
43
Response Rate = • 56% Study Design = • Cross-
sectional
tion, trust, and workplace cohesiveness
Personal = • Values
align with leadership
Edwards et al., 2018
Investigate relationship between burnout in primary care workers and characteristics of primary care practices
PCP sample • Physicians
(n = 1535) • NPs/PAs
(n = 946) Primary Care practices located in = • Oregon,
Washington, Wisconsin, Idaho, Illinois, Indiana, Colorado, New York, New Mexico, North Caroline, Virginia, and Oklahoma
Response Rate = • 87% Study Design = • Cross-
sectional
Rohland et al., (2004) single item burnout measure
Organizational = • Practice
size • Type of
health care system
• Affiliation with ACO
• Job role • Longer
hours worked per week
Personal = • Longer
tenure
Outcomes = • NR Prevalence = • 25.1% in
physicians • 22.6% in
NPs/PAs
Yoon et al., 2017
Assess relationship between a sense of calling with primary care physician and
PCP sample = • Internal
medicine, family medicine, and general practice physicians
Rohland et al., (2004) single item burnout measure
Personal = • Sense of
Calling
Outcomes = • NR Prevalence = • 22.8% in
primary care physicians
44
psychiatrists burnout, well-being, and commitment to clinical practice
(n = 896) Setting = • Specific
states in the U.S. NR
Response Rate = • 63% Study Design = • Cross-
sectional Ratanawongs et al., 2008
Explore relationship between primary care physician burnout and patient-physician communication
PCP sample = • Physicians
(n = 40) Setting = • Baltimore,
Maryland Response Rate = • NR Study Design= • Longitudina
l cohort study
MBI Organizational = • Patient
insurance status
Patient = • Provider-
patient communication
Prevalence = • 38% High
Burnout • 27%
Medium Burnout
• 35% Low Burnout
Williams et al., 2007
Identify the conditions that are associated with physician burnout, stress, and dissatisfaction and whether these outcomes result in the delivery of poor quality care
PCP sample = • Physicians
(n = 426) Setting = • New York
City, New York; Chicago, Illinois; Milwaukee, Madison, and smaller towns in Wisconsin
Response Rate = • Ranged
from 70% to 100% across different settings
Study Design =
Rohland et al., (2004) single item burnout measure
Organizational = • Job stress Personal = • Job
satisfaction
Patient = • Self-
reported likelihood of error
• Suboptimal patient care**
Prevalence = • NR
45
• Cross-sectional
Kim et al., 2018
Examine whether not relying on team members to perform primary care tasks by PCPs is associated with PCP burnout
PCP sample = • Physicians,
NPs, & PAs (n = 327)
Setting = • VA primary
care practices in California and Nevada
Response Rate = • 54% in
wave 1 of the survey and 39% in wave 2 of the survey
Study Design = • Cross-
sectional
MBI Organizational = • Not
working in teams to provide patient education on disease specific self-care activities and lifestyle factors
• Team communication
Outcomes = • NR Prevalence = • NR
Willard-Grace et al., 2019
Determine whether PCP burnout and low employee engagement predicts PCP turnover
PCP sample = • Physicians
(n = 199) • NP/PAs
(n = 53) Setting = • San
Francisco, California
Response Rate = • 90% Study Design = • Longitudina
l cohort study
MBI-General Survey
Predictors = • NR
Organizational = • Turnover Prevalence = • 53% in
PCPs
Zubatsky et al., 2018
Determine whether higher integration of behavioral
PCP sample = • Physicians
(n = 288) Setting = • National
sample with
MBI Organizational = • Behavioral
health integration
Outcomes = • NR Prevalence = • NR
46
health services in primary care is associated with lower burnout among primary care physicians
specific states NR
Response Rate = • 91.5% Study Design = • Cross-
sectional
in primary care
Personal = • Gender • Shorter
tenure
Sun et al., 2017
Investigate the association between primary care physician burnout and empathy with antibiotic prescribing for patients with acute respiratory infections
PCP sample = • Physicians
(n = 102) Setting = • Primary
care practices within the Cleveland Clinic Health System in Ohio
Response Rate = • NR Study Design = • Cross-
sectional
MBI Predictors = • NR
Patient = • No
association between physician burnout and antibiotic prescribing
Prevalence = • NR
Whitebird et al., 2017
To describe PCP’s feelings of burnout, job satisfaction, and satisfaction with resources needed for caring for complex patients
PCP sample = • Physicians
(n = 578) • NPs
(n = 56) • PAs
(n = 57) Setting = • Minnesota,
Colorado, California, Washington, Michigan, Pennsylvania, Florida, and Massachusetts
Rohland et al., (2004) single item burnout measure
Personal = • Longer
tenure • Career
dissatisfaction
• Lower PCP perceptions of being able to care for complex patients
• Lower PCP satisfaction with resources for treating
Outcomes = • NR Prevalence = • 31%
47
Response Rate = • 45.6 % Study Design = • Cross-
sectional
complex patients
Helfrich et al., 2017
Examine the association between staffing and workload with burnout among primary care workers
PCP sample = • Physicians,
NPs, & PAs (n = 1517)
Setting = • Veterans
Affairs primary care clinics in the U.S.
Response Rate = • 20.9 % Study Design = • Cross-
sectional
Dolan’s et al., (2015) single item burnout measure
Organizational = • Inadequate
staffing • Staff
turnover • Workload
(i.e., patient panel overcapacity)
• Working extended hours during weekends
• Working extended hours with team
Outcomes = • NR Prevalence = • 49.2% in
PCPs
Note. NPs = Nurse Practitioners; PAs = Physician Assistants; NR = Not Reported; EHRs = Electronic Health Records; ACO = Accountable Care Organization; PCP = Primary Care Provider; *Adaptive reserve score is defined as a measure of the organization’s leadership, communication practices, trust and teamwork, work culture, and collective efficacy. The higher the adaptive reserve score, the greater the organization’s ability to implement change; ^Work environment domain consists of workload, control, fairness, values, rewards, and a sense of community; MBI = Maslach Burnout Inventory consists of the emotional exhaustion, depersonalization, and personal accomplishment subscales; MBI-General Survey consists of the exhaustion, cynicism, and professional efficacy subscales; PACT = Patient Aligned Care Team; **Suboptimal patient care is defined as discharging patients to make care delivery feasible, not fully discussing treatment options or answering patient questions, making treatment or medication errors, ordering medications without assessing patients, and skipping procedures to facilitate early patient discharge; Study authors who reported using a single item burnout measure, all used the same single item regardless of citing Dolan et al., 2015 or Rohland et al., 2004.
48
2.21 Table 2
49
2.22 Table 3
Quality Appraisal of the Cohort Studies (n = 2)
Studies Appraisal Questions Ratanawongs et
al., 2008 Willard-Grace et al., 2019
Q1 Were the two groups similar and recruited from the same population?
• •
Q2 Were the exposures measured similarly to assign people to both exposed and unexposed groups?
? •
Q3 Was the exposure measured in a valid and reliable way?
• •
Q4 Were confounding factors identified? • • Q5 Were strategies to deal with confounding factors
stated? • •
Q6 Were the groups/participants free of the outcome at the start of the study (or at the moment of exposure)?
? •
Q7 Were the outcomes measured in a valid and reliable way?
• •
Q8 Was the follow up time reported and sufficient to be long enough for outcomes to occur?
• •
Q9 Was follow up complete, and if not, were the reasons to loss to follow up described and explored?
? •
Q10 Were strategies to address incomplete follow up utilized?
° •
Q11 Was appropriate statistical analysis used? • • Note. • = Yes; ° = No; ? = Unclear; N/A = Not Applicable; Q = Question;
50
2.23 Table 4
Predictors associated with PCP Burnout
Predictors Association with Burnout
Poor work control ↑ Time pressure ↑ High workload/demands ↑ Task discordance ↑ Patient panel overcapacity ↑ Chaotic practice environments ↑ Poor staffing ↑ Staff turnover ↑ Full-time employment ↑ Working > 40 hours/week ↑ Not working in teams ↑ Certain patient care tasks ↑ Use of EHRs ↑ Greater out-of-work hours spent using the EHR ↑ PCPs in accountable care organizations ↑ Working extended hours during weekends ↑ Career dissatisfaction ↑ Lower PCP satisfaction with resources needed to care for complex patients ↑ Lower PCP perceptions of being able to care for complex patients ↑ Certain types of primary care practices (e.g., Non-solo primary care practices, Hospital/Health system owned practice, & Federally Qualified Health Centers)
↑
Availability of clinic resources to address patient needs ↓ Task delegation ↓ Presence of a PACT coach ↓ Sense of calling ↓ Consistently working with the same staff ↓ Positive team culture ↓ Tighter team structure ↓ Participatory decision making ↓ Effective team communication ↓ Higher levels of behavioral health integration in primary care ↓ Alignment of personal values with administration ↓ Strong emphasis on leadership, communication, trust, & teamwork within practices
↓
Length of tenure * Sex *
Note. “↑” increases burnout; “↓” decreases burnout; “*” mixed; EHR = electronic health record; PCP = primary care provider; PACT = Patient Aligned Care Team;
51
2.24 Table 5
PCP burnout and associated outcomes
Outcomes Association with PCP burnout Lower quality of patient care * Medical errors * Lower job satisfaction Y High job stress Y PCPs leaving the practice (i.e., turnover) Y Lower patient satisfaction N Antibiotic prescribing for patients with ARI N
Note. “Y” yes, associated with PCP burnout; “*” mixed whether there is an association with PCP burnout; “N” not associated with PCP burnout; PCP = primary care provider; ARI = Acute respiratory infections;
52
Chapter 3: Primary Care Practice Environment and Burnout
Among Nurse Practitioners
In this chapter, the author investigates whether the primary care practice environment is
associated with NP burnout. This manuscript has been submitted to a peer reviewed journal.
53
3.1 Introduction
Clinician burnout is recognized by feelings of emotional exhaustion, depersonalization,
and low personal accomplishment (Maslach & Jackson, 1981). Nearly 50% of pediatric critical
care physicians (Shenoi et al., 2018), 40.1% of transplant surgeons (Jesse, Abouljoud, &
Eshelman, 2015), and 35.3% of nurses (Dyrbye et al., 2019) are burnt-out. Given the high
prevalence of burnout, researchers have classified burnout as a public health crisis (Jha et al.,
2019). Burnout has gained national attention as it leads to lower clinician productivity, job
dissatisfaction, and increases clinician’s intention to leave their job (West et al., 2018; Dewa et
al., 2014; Shanafelt et al., 2015). Clinicians experiencing burnout are also more likely to be
involved in adverse events (Panagioti et al., 2018) such as medical errors (Dewa et al., 2017) and
health care associated infections (Cimiotti et al., 2012).
To date, an extensive body of research exists on clinician burnout in the acute care setting
(Clark & Lake, 2020; Shenoi et al., 2018; Jesse et al., 2015; Bragard et al., 2015; Dewa et al.,
2017; Cimiotti et al., 2012). Such researchers frequently suggest that a poor practice environment
within the health care setting contributes to clinician burnout (McHugh et al., 2011; Jha et al.,
2019; National Academy of Medicine, 2018; Van Bogaert et al., 2013). In one study of hospital
nurses, researchers found that low autonomy to make patient care decisions, multiple job
demands, and limited support from administrators leads to burnout (McHugh et al., 2011). In a
meta-analysis with 17 studies, hospital nurses in better work environments had 26% lower odds
of burnout (Lake et al., 2019), thus indicating that the environment where nurses deliver patient
care can influence burnout. In additional studies focusing on physicians, researchers have found
that a poor practice environment such as high workload and inefficient working relations with
colleagues were significantly associated with burnout (Bragard et al., 2015; West et al., 2016;
54
Olson et al., 2018). While most researchers have focused on burnout in the acute care setting
(Bragard et al., 2015; Dewa et al., 2017; McHugh et al., 2011; Cimiotti et al., 2012; O’Mahony,
2011), little attention has been given to burnout among primary care providers (PCPs) including
physicians, nurse practitioners (NPs), and physician assistants. Given that most health care
services are delivered by clinicians in primary care practices (Petterson et al., 2018), the
investigation of clinician burnout in the primary care setting is crucial.
Among all health care providers, PCPs have one of the highest rates of burnout (48%)
(Edwards et al., 2018), and researchers report that poor practice environments are a common
modifiable predictor of burnout (Abraham, Zheng, Poghosyan, 2019; Willard-Grace et al., 2019;
Rabatin et al., 2016; Babbott et al., 2014; Edwards et al., 2018; Helfrich et al., 2014; Blechter et
al., 2017; Spinelli et al., 2016; Olayiwola et al., 2018). Yet, of the studies investigating burnout
among PCPs, most explore primary care physician burnout (Rabatin et al., 2016; Babbott et al.,
2014; Linzer et al., 2009). As a result, the prevalence and factors associated with burnout among
other types of PCPs, specifically NPs, remains greatly understudied.
Currently, 69% of NPs deliver primary care services to patients (AANP, 2020), and by
2030 it is projected that NPs will comprise about one-third of all PCPs (U.S. Department of
Health and Human Services, 2016). Primary care NPs are nurses with additional training at the
Masters or Doctoral level and can assess, diagnose, and manage patient conditions, order and
interpret tests, prescribe medications, and provide patient education (AANP, 2020). Compared to
other PCPs, NPs are more likely to practice in low-income, high-minority practices (Barnes et
al., 2018; Buerhaus et al., 2015) which generally have limited resources needed for patient care
(Chan et al., 2019). Furthermore, 46.1% of NPs reported working in a poor practice environment
(Carthon et al., 2020), and, in another study, one in every four NP felt that their role was not
55
well-understood by practice administrators (Poghosyan & Aiken, 2015). In addition, only 39.5%
of NPs reported that practice administrators treat NPs and physicians equally (Poghosyan &
Aiken, 2015), suggesting that possibly more than half of the NPs felt otherwise. Moreover, NPs
in poor practice environments had limited administrative support and resources, strained
relations with physicians, poor communication with staff, and limited independence (Poghosyan
et al., 2013). As a result, challenges within the primary care practice environment may
predispose NPs to burnout. The purpose of this study is to assess burnout among primary care
NPs and investigate whether the primary care NP practice environment is associated with NP
burnout.
3.11 Methods
Conceptual Framework: The Clinician Well-Being and Resilience model (Brigham et
al., 2018) guided this study (Figure 1). The model was developed by researchers at the National
Academy of Medicine and illustrates the relationship between internal and external factors
contributing to burnout and the consequences of burnout (Brigham et al., 2018). We adapted the
model to focus specifically on the relationship between primary care NP practice environment,
burnout, NP demographic characteristics (i.e., age, sex, education, race, marital status, hours
worked per week, years of NP experience, workload, and years employed in the current
practice), and characteristics of the primary care practice (i.e., size, hours of operation, and type
of practice).
Study Design and Data Sources: This study is a secondary analysis of cross-sectional
survey data obtained from a large study (Poghosyan et al., 2020) focused on health disparities for
older adults with chronic conditions receiving primary care services from NPs.
Description of the Parent Study: Participants & Setting: Researchers discerned primary
56
care NPs from the SK&A OneKey primary care practice database which contains information on
practicing primary care NPs (DesRoches et al., 2015). NPs practicing in any one of the following
three types of primary care practices were included; 1) an independent solo practice with the
physician specializing in primary care and a minimum of one NP working in that practice, 2) a
medical group practice where the entire practice focuses on primary care and a minimum of one
NP working in that practice, or 3) a medical group practice employing one or more physicians
and which has at-least 50% of the physicians specializing in primary care with a minimum of one
NP working in that practice. Considering that there were more NPs in larger states (i.e.,
California, Texas, and Pennsylvania) than in smaller states (i.e., Washington, New Jersey, and
Arizona), researchers used a random sample of NPs to create comparable counts of NPs across
the states included in the sample.
Data Collection: Primary care NPs completed surveys asking about their practice
environment, burnout, demographics, and characteristics of their main practice site. Using a
modified Dillman approach for mixed-mode surveys (Dillman, Smyth, & Christian, 2014),
researchers sent NPs mailed surveys which also contained an online link to enable NPs to
complete the surveys electronically or by paper. In total, three rounds of mailed questionnaires
and two postcard reminders were sent to non-responding NPs. To increase the overall response
rate, researchers conducted follow up calls to non-responding NPs. In these calls, it was
determined that some NPs were ineligible because they never worked in the practice or no longer
worked in the practice. Other NPs were labeled as “unknown” because, despite researchers
calling those practices three separate times, no one answered to allow researchers to verify NP
eligibility. Ultimately, 1,244 NPs from six states completed the survey. Response rates were
determined using three different scenarios. In one scenario, it was assumed that all non-
57
responding NPs were eligible and this resulted in the most conservative response rate of 22.2%.
In a second scenario, it was assumed that almost half of the “unknown,” non-responding NPs
were ineligible, and this resulted in a 25.7% response rate. In the third scenario, it was assumed
that all of the non-responding NPs were ineligible, resulting in a 31.8% response rate. Thus, the
true response rate may range from 22.2% to 31.8%.
Present Study: For this secondary data analysis, we had a final sample of 396 primary
care NPs practicing in either New Jersey or Pennsylvania. These two states were chosen because
they have comparable measures of access to care (U.S. News & World Reports, 2019) and
quality of care (U.S. News & World Reports, 2019), and so NPs practicing in these states would
be working in comparable health care markets. Approval from the institutional review board of a
large university located in the Northeast was received prior to conducting this study.
Measures: Practice Environment: To measure the practice environment, the Nurse
Practitioner Primary Care Organizational Climate Questionnaire (NP-PCOCQ) was used
(Poghosyan et al., 2013). The NP-PCOCQ is a 29-item survey which has four subscales
measuring the following domains: NP-Physician Relations (NP-PR), Professional Visibility
(PV), NP-Administration Relations (NP-AR), and Independent Practice and Support (IPS)
(Poghosyan et al., 2013). The NP-PR subscale (7 items) measured the degree of teamwork
between NPs and physicians, and it also measured whether physicians trust the decisions made
by NPs regarding patient care. The PV subscale (4 items) measured the clarity and visibility of
the NP role within their practice. The NP-AR subscale (9 items) measured NP’s perception of
relations with administrators from their practice, and the IPS subscale (9 items) measured the
availability of resources and support NPs have in their practice. The NP-PCOCQ is widely used
across the U.S. and abroad (Poghosyan et al., 2017; Scanlon et al., 2018; Poghosyan et al., 2015;
58
Rowand, 2017; Haupt, 2016) and has strong psychometric properties. This tool has been tested
for content, construct, structural, discriminant, and predictive validity (Poghosyan et al., 2013).
In this present sample of primary care NPs from New Jersey and Pennsylvania, the Cronbach’s
alpha for the NP-AR subscale was 0.94, the NP-PR subscale was 0.88, the PV subscale was 0.85,
and the IPS subscale was 0.87.
The NP-PCOCQ uses a 4-point Likert scale that ranges from “strongly agree” to
“strongly disagree” (Poghosyan et al., 2013). Higher mean subscale scores for each individual
NP-PCOCQ subscale indicated better primary care NP practice environments. Since the practice
environment is a shared perception held by NPs, organizational level scores were generated by
aggregating the individual practice environment scores of NPs in the same practice for each of
the four NP-PCOCQ subscales.
Burnout: Primary care NP burnout, as the dependent variable, was measured using a
widely used non-proprietary, single item measure (Dolan et al., 2015). One of its earliest use was
in a study investigating the association between physician outcomes and health maintenance
organizations (Dolan et al., 2015). In a national sample of primary care clinicians, this non-
proprietary single item burnout measure was compared against a standalone item from the
emotional exhaustion subscale of the Maslach Burnout Inventory (Dolan et al., 2015).
Researchers concluded that the prevalence of burnout was relatively comparable when using the
non-proprietary single item measure (38.5%) versus the single item from the emotional
exhaustion subscale of the Maslach Burnout Inventory (36.7%) (Dolan et al., 2015). Another
advantage of using the non-proprietary single item measure is that it is cost-free and less
burdensome for participants (Dolan et al., 2015). For our study, NPs self-reported their level of
burnout from one (I enjoy my work. I have no symptoms of burnout) to five (I feel completely
59
burned-out and often wonder if I can go on). A higher score indicated greater burnout (Dolan et
al., 2015). For the ease of interpretation, we dichotomized responses so that scores ranging from
three to five were combined to indicate “burnout,” and scores one to two were combined to
indicate “no burnout.” This dichotomization of the burnout variable has been commonly used in
prior studies investigating PCP burnout (Helfrich et al., 2014; Edwards et al., 2018).
Covariates: We retrieved data on characteristics of NPs (i.e., age, sex, race, education,
marital status, workload, years employed in current practice, number of hours worked per week,
and years of NP experience) and characteristics of the main practice site (i.e., size, hours of
operation, and type of primary care practice).
Statistical Analysis: All data were cleaned and coded in SPSS statistical software (IBM
Corp, 2017). The distribution of continuous variables was inspected for outliers using boxplots
and any outlier values from the dataset were removed. We used the variance inflation factor
(VIF) statistics to check for multicollinearity, and VIF values less than five suggested lack of
multicollinearity (Akinwande, Dikko, Samson, 2015).
Multiple Imputation: Given that certain variables in our study had up to 12% of missing
observations, we used multiple imputation analyses because it is a methodologically rigorous
approach to handling missing data and it is recommended over traditional techniques such as
mean substitution (Murray, 2018; McCleary, 2002). We created 10 simulated versions of the
dataset based on the percentage of missing data (Bodner, 2008). Those datasets were analyzed,
combined, and adjusted to generate pooled results which represented the aggregated scores from
the 10-simulated versions of the dataset (Murray, 2018; McCleary, 2002). We extracted and
imported the pooled dataset into STATA statistical software (StataCorp LP, 2015).
60
Multi-level Analyses: We estimated risk ratio (RR) rather than odds ratio using four
multi-level cox regression models with time as a constant variable. This was done because the
prevalence of NP burnout was greater than 10%, and researchers have found that logistic
regression models estimating odds ratio may overestimate the results when the prevalence of an
outcome is more than 10% (Diaz-Quijano, 2012). While cox regression models report the hazard
ratio (Stare & Maucort-Boulch, 2016), the use of a constant time variable makes the value of the
hazard ratio the same as the RR (Diaz-Quijano, 2012). Thus, throughout, we use the term “risk
ratio” instead of “hazard ratio.” To illustrate the strength and direction of the relationship
between the four practice-level NP-PCOCQ practice environment subscales and burnout, we
reported the 95% confidence intervals (CI), the p-values, and the RRs. The four multi-level cox
regression models include NP-level demographic variables in level one of the model as well as
survey modality (i.e. mail or online), and NP burnout. Variables measuring the practice
environment, practice size, main practice site, and hours of operation were practice-level
variables within level two of the model. We included state as a fixed-effect variable.
3.12 Results
This study used data from 396 NPs, with 27.5% of NPs being from New Jersey and the
rest from Pennsylvania (Table 1). Most NPs were female (90.4%) and White (89.4%). The
average age of these NPs was 49.5 years (Standard Deviation [SD] = 12.0 years). Only 3% of
NPs had a PhD or other doctorate as their highest educational degree, 7.1% held a Doctorate of
Nursing Practice (DNP), and 87.6% had a Master’s degree. On average, NPs worked 38.9 hours
per week (SD = 11.2) with most of their time spent on providing direct clinical patient care,
followed by coordinating patient care. Most primary care NPs (60.4%) practiced in a physician-
owned clinic. The practice-level scores on the NP-PCOCQ subscales ranged from 2.82 (SD =
61
0.71) on NP-AR to 3.42 (SD = 0.48) on IPS. The practice-level score on the PV subscale was
3.11 (SD = 0.65), and 3.33 (SD = 0.52) for the NP-PR subscale.
Overall, 25.3% of NPs were burnt-out. Multicollinearity was not discovered in any of the
final multi-level cox regression models (mean VIF = 1.70). In our multi-level models in which
time was a constant variable (Table 2), higher practice-level PV subscale scores were associated
with 51% lower risk of NP burnout (RR = .49, 95% CI = .36 to .66). Similarly, higher practice-
level NP-PR subscale scores were associated with 51% lower risk of NP burnout (RR = .49, 95%
CI = .34 to .71). Higher practice-level NP-AR subscale scores were associated with 58% lower
risk of NP burnout (RR = .42, 95% CI = .31 to .56), and higher practice-level IPS subscale scores
were associated with 56% lower risk of NP burnout (RR = .44, 95% CI = .29 to .65), after
controlling for provider and other practice-level characteristics.
3.13 Discussion
In this study, we examine the association between the primary care practice environment
and burnout among NPs in New Jersey and Pennsylvania. We found that the prevalence of
primary care NP burnout is 25.3% which is slightly higher than the combined burnout prevalence
of 22.6% among primary care NPs and physician assistants (Edwards et al., 2018). Burnout
among primary care NPs is concerning as it may affect the care that is delivered to patients.
However, across all subscales, when NPs practice in favorable environments, the risk of them
reporting burnout significantly decreases anywhere from 51% to 58%.
In particular, higher mean NP-PR, IPS, PV, and NP-AR subscale scores are associated
with 51%, 56%, 51%, and 58% lower risk of NP burnout, respectively. These findings suggest
that NP burnout decreases when NPs practice in environments that promote optimal relations
with physicians and administrators, administrators have an understanding of NP role, skills, and
62
competencies, and NPs are supported and provided resources needed to perform their jobs.
Moreover, our findings are consistent with a recent study which found that nurses working in
environments with strong collaboration, greater communication and effective decision making,
and supportive administrators reported lower burnout (Kim et al., 2020). Thus, our study and
existing studies (Kim et al., 2020; Lake et al., 2019) highlight the important role of the practice
environment in influencing burnout among nurses.
However, of the four practice environment subscales, NPs reported that their relations
with practice administrators was the lowest. This finding is consistent with a prior qualitative
study reporting that most NPs felt that administration was not supportive nor understanding of
the contributions that NPs make to patient care, and in some cases, NPs were not represented
within the administration (Poghosyan et al., 2013). Although our study did not explore why NP-
AR was suboptimal, it is important to investigate the relations between NPs and practice
administrators since having favorable relationships with management may be important for
ultimately reducing burnout.
Implications: The results of this study provide implications for clinical practice. Since
NPs are increasingly delivering patient care in primary care practices and play a vital role in the
health care system, it is crucial to reduce NP burnout. With strong links already reported between
provider burnout and adverse patient and organizational outcomes (West et al., 2018), it is
essential that clinicians and administrators in primary care practices make reducing NP burnout
an organizational priority. One way for clinicians and practice administrators to reduce NP
burnout is by fostering the development of a healthy primary care practice environment. For
example, in a qualitative study involving primary care workers (e.g., physicians, nurses, assistant
nurses, administrative staff) in Sweden, researchers found that healthy primary care practice
63
environments consisted of primary care workers practicing in an organization with positive,
accessible, and fair leaders who encouraged collaboration, teamwork, empowerment, and
autonomy (Josefsson et al., 2018). Additionally, consistent with our findings, Josefsson and
colleagues (2018) reported that visibility and recognition of clinician role, was also a feature of a
healthy work environment. Based on the results from our study, it is possible that creating a
healthy practice environment may hold much potential for alleviating NP burnout.
In addition to practice implications, we also have implications for future research. Since
NPs reported that their relations with administrators was least favorable when compared with the
other three practice environment subscales, future research should investigate what interventions
are successful in improving relations between NPs and administrators. Furthermore, more
research is needed to examine how the relations between primary care NPs and practice
administrators are related to patient (e.g., quality of care, patient satisfaction) and organizational
outcomes (e.g., turnover). Lastly, future research should be conducted with all PCPs (i.e.,
physicians, NPs, and physician assistants) which will allow researchers to better understand how
burnout impacts all PCPs and the quality of care delivered to patients in primary care settings
across the U.S.
Limitations: There are limitations in this study. The generalizability of study findings is
limited because our sample stemmed from two states: New Jersey and Pennsylvania. Thus, the
findings are limited to only primary care NPs from these two states. Since this study relied on
self-report data, bias may be an issue. Despite best attempts to encourage NPs to complete the
survey, the low response rate may suggest that the NPs in our study are unlike NPs nationally.
However, we found that NPs in our study were relatively alike in race (89.4% White vs. 87%
White nationally) and age (49.5 years vs. 49 years nationally) to NPs from the National Nurse
64
Practitioner Sample Survey (AANP, 2019). Furthermore, we cannot conclusively say poor
practice environments cause NP burnout since we had cross-sectional survey data and such data
limits causal inference. Despite these limitations, our study contributes to the growing literature
on provider burnout by identifying the current prevalence of primary care NP burnout and a key
variable, the practice environment, that may contribute to their burnout.
3.14 Conclusion
To our knowledge, this is the first study investigating the association between the primary
care practice environment and NP burnout. This study provides valuable evidence and supports
existing literature by showing that favorable NP practice environments have the potential for
reducing primary care NP burnout. Additional research examining the association between NP
practice environments with outcomes such as NP intention to leave the practice and quality of
patient care are recommended.
65
3.15 Figure 1
Adapted Clinician Well-Being and Resilience Model
Note. NP = Nurse Practitioner;
66
3.16 Table 1
Characteristics of Primary Care Nurse Practitioners (n = 396)
N (%) Mean (SD) Age 49.5 years (12.0) Female 358 (90.4) Highest Educational Degree Master’s degree 347 (87.6) Doctorate of Nursing Practice (DNP) 28 (7.1) PhD or other doctorate 12 (3.0) Other 9 (2.3) Race White 354 (89.4) Hispanic/Latino 12 (3.0) Marital status Married 315 (79.5) Years of NP experience 11.4 years (8.9) Years employed in current practice 6.4 years (6.3) Average number of hours worked per week 38.9 hours (11.2) NP workload Providing direct patient care 31.1 hours (10.1) Coordinating patient care 5.6 hours (6.0) Providing care management services 2.0 hours (3.3)
Performing quality assurance and improvement activities
1.6 hours (3.5)
Conducting administrative activities/leadership 1.7 hours (3.9) Survey Modality Mail 309 (78.0) Online 87 (22.0) Burnout 100 (25.3) Characteristics of Primary Care Practices Practice Location New Jersey 109 (27.5) Pennsylvania 287 (72.5) Main practice site Physician practice 239 (60.4) Community health center 51 (12.9) Hospital based clinic 37 (9.3) Retail-based clinic 8 (2.0) Urgent care clinic 12 (3.0) Nurse managed clinic 4 (1.0) Other 45 (11.4) Practice Open on Weekends 141 (35.6)
67
Practice Open at Night None 127 (32.1) Once a week 78 (19.7) Twice a week 82 (20.7) Three times a week 24 (6.1) Four or more times a week 85 (21.5) Primary Care NP Practice Environment Scores Professional Visibility 3.11 (0.65) NP-Administration Relations 2.82 (0.71) NP-Physician Relations 3.33 (0.52) Independent Practice and Support 3.42 (0.48)
Note. SD = Standard Deviation; NP = Nurse Practitioners; Mean scores on the primary care NP practice environment subscales range from 1 to 4, with higher score indicating more favorable practice environments for NPs;
68
3.17 Table 2
Results of multi-level cox regression models, with time as a constant variable, assessing the
association between practice environment subscales and NP burnout (N = 396)
Note. NP = Nurse Practitioner; *Because we did not have NP responses recorded over various time points, we created a constant time variable. When there is a constant time variable in a multi-level cox regression model, the hazard ratio is the same as the risk ratio. Thus, we reported the risk ratio in this table; Four independent multi-level models were run, one for each practice environment subscale. a In each model, we controlled for age, sex, race, education, marital status, workload, years of experience as an NP, average hours worked per week, practice size, hours of operation (nights and weekends), type of primary care practice, survey type, and state.
Modela Practice environment subscale Risk Ratio*
P > |z| 95% Confidence Interval
1 Professional Visibility 0.49 0.00 0.36 0.66 2 Independent Practice and
Support 0.44 0.00 0.29 0.65
3 NP-Physician Relations 0.49 0.00 0.34 0.71 4 NP-Administration Relations 0.42 0.00 0.31 0.56
69
Chapter 4: Use of Multifunctional Electronic Health Records and
Burnout among Primary Care Nurse Practitioners
In this chapter, the author investigates whether use of multifunctional Electronic Health
Records (EHRs) is associated with primary care NP burnout. This manuscript has been submitted
to a peer reviewed journal.
70
4.1 Introduction
In the United States (U.S.), nearly 48% of primary care providers (PCPs) including
physicians, nurse practitioners (NPs), and physician assistants are experiencing burnout
(Edwards et al., 2018). Burnout is described as an internalized feeling of emotional exhaustion,
depersonalization, and feelings of low personal accomplishment (Maslach & Jackson, 1981). In
2018, the American Public Health Association identified burnout as a significant public health
problem (Krisberg, 2018) and accumulating evidence shows that burnout negatively impacts
clinicians, patients, and the entire health care system (West et al., 2018). Clinician burnout can
lead to medical errors (West et al., 2018), increased risk of motor vehicle accidents and
suicidality (Patel et al., 2018), and increase intention to leave their job (Abraham et al., 2019;
West et al., 2018; Dewa et al., 2014).
There are several known factors that contribute to clinician burnout including high
workloads, poor staffing, long work hours, poor practice environments, job stress, loss of
workplace autonomy, and limited support and resources to provide patient care (Abraham et al.,
2019; West et al., 2018). A large body of research shows that widespread use of electronic health
records (EHRs) is emerging as another major source of burnout for PCPs (Willard-Grace et al.,
2019; Abraham et al., 2019; Babbott et al., 2014). EHRs in primary care practices include
reminder systems and multiple computerized features allowing for patient medication updates,
use of drug-related alerts, prescription ordering, and documentation of clinical notes during a
patient visit (Huang et al., 2018). In a national study involving physicians, those who used EHRs
and computerized physician order entries were less satisfied with the amount of time they spent
on clerical tasks and were at greater risk for burnout (Shanafelt et al., 2016). Furthermore, 62.5%
of physicians felt that the EHR did not improve their efficiency in providing patient care
71
(Shanafelt et al., 2016), which is concerning because inefficiencies within practices can lead to
physician burnout (Panagioti, Geraghty, & Johnson, 2018).
Since the passage of the Health Information Technology for Economic and Clinical
Health Act in 2009, use of EHRs in primary care practices increased from 25% to nearly 90%
(Gordon, Baier, & Gardner, 2015). Providers increasingly use EHRs with multiple features, poor
usability and interoperability (Howe et al., 2018), and report that using these multifunctional
EHRs contributes to their feelings of burnout (Robertson et al., 2017; Babbott et al., 2014). In
particular, primary care physicians report greater burnout in practices with more EHR features
such as electronic alerts and computerized capabilities (e.g., electronic medication prescribing,
laboratory requests and results, radiology reports and images, referrals to specialists, discharge
summaries, electronic messaging to and from patients) (Babbott et al., 2014). As a result, 75% of
primary care physicians attribute their feelings of burnout to EHR use (Robertson et al., 2017).
One reason why the EHR may contribute to physician burnout is because of the increased time
physicians spend on the EHR. For example, in a multi-site study with physicians in ambulatory
facilities across Illinois, New Hampshire, Virginia, and Washington, physicians spent an
additional two hours on the EHR for every hour that they provided direct clinical care to patients
(Sinsky et al., 2016). The increased time physicians spend on the EHR may stem from the
complex features within the EHR, such as computerized physician order entries, patient portals,
medical documentation requirements, and management of one’s inbox (Arndt et al., 2017), and
can increase risk for burnout.
Overall, use of multifunctional EHRs containing various computerized capabilities and
poor usability or operability contribute to primary care physician burnout (Rabatin et al., 2016;
Babbott et al., 2014; Edwards et al., 2018; Helfrich et al., 2014; Linzer et al., 2009). However, it
72
is unknown whether use of multifunctional EHRs is associated with burnout in primary care NPs
who, in 2016, represent about 25.2% of the primary care workforce in rural practices and 23.0%
in non-rural practices (Barnes et al., 2018; Pohl et al., 2018). Primary care NPs have obtained
additional education at the Masters or at the Doctoral level and are qualified to assess, diagnose,
and treat patients, manage chronic conditions, order tests and prescribe medications, and provide
health and wellness to those across the lifespan (AANP, 2020). In the U.S., 69% of practicing
NPs deliver primary care services to patients (AANP, 2020). Compared to all PCPs, primary care
NPs have the fastest growing PCP workforce with an expected growth of 93% from 2013 to
2025 in the U.S. (U.S. Department of Health and Human Services, 2016).
In addition to a growing NP workforce, NPs are using EHRs, which is a known predictor
of clinician burnout (West et al., 2018), but it is unknown whether using multifunctional EHRs is
a contributor of primary care NP burnout. With NPs delivering a significant portion of primary
care to patients, if the EHR is associated with NP burnout, interventions can be designed to
improve the usability and operability of EHRs in primary care practices, thus promoting NP
well-being and improving patient care. The purpose of this study is to determine whether use of
multifunctional EHRs is associated with NP burnout in primary care practices.
4.11 Methods
Conceptual Framework: The study is guided by the Clinician Well-Being and
Resilience model (Brigham et al., 2018) (Figure 1). Researchers at the National Academy of
Medicine developed the model to show the factors associated with clinician burnout and well-
being across all health care professions (Brigham et al., 2018). For this study, we adapted the
model to focus on the relationship between use of multifunctional EHRs and primary care NP
burnout while also considering NP and practice characteristics. Use of multifunctional EHRs is
73
conceptualized as computerized capabilities and electronic reminder systems for decision support
of clinical guidelines, which exist within the primary care practice (Friedberg et al., 2009). The
adapted conceptual model also includes characteristics describing NPs such as age, sex,
education, marital status, hours worked per week, workload, experience, and years employed in
the current practice. Characteristics of the primary care practice include size, type of practice,
and hours of operation.
Study Design and Data Source: This is a secondary analysis of cross-sectional survey
data collected from the parent study (Poghosyan et al., 2020) which is described below.
Description of the Parent study: Participant & Setting: Researchers for the parent study
identified primary care NPs from the SK&A OneKey primary care practice database which
contains information on NPs in office-based practices as well as practices owned by or operated
in hospitals (DesRoches et al., 2015). To recruit NPs in primary care practices, researchers
obtained data on three types of primary care practices: 1) Independent solo practitioner practices
in which a physician had a specialization of primary care (i.e., family medicine, general practice,
internal medicine, internal medicine/pediatrics, internal medicine/preventive medicine, general
preventive medicine, or geriatrics) and at-least one NP worked in the practice. 2) Medical group
practice sites in which the entire facility specialized in primary care and at least one NP worked
in that practice. 3) Medical group practice sites with one or more physician, and that
encompassed at-least 50% of physicians having the specialty of “primary care,” and at-least one
NP worked in that practice. Since the number of NPs in larger states (i.e., California,
Pennsylvania, Texas) is greater than the number of NPs in smaller states (i.e., New Jersey,
Arizona, Washington), a random sampling of NPs was conducted to allow for similar numbers of
NPs across states.
74
Data Collection: From 2018-2019, primary care NPs completed surveys containing
measures of their use of the EHR, burnout, demographics, and characteristics of their main
practice site. To collect data, researchers used a modified Dillman approach for mixed-mode
surveys (Dillman et al., 2014), in which the mailings contained an online survey link to allow
NPs to complete the survey electronically or on paper. In total, three mailed questionnaires were
sent to NPs followed by two rounds of reminder postcards to non-responding NPs two weeks
after the first and second mailings. To obtain a higher overall NP response rate, follow-up phone
calls were made to the non-responding NPs. In these calls, some NPs were deemed ineligible
(e.g., never worked or no longer worked in the practice) and others were labeled as “unknown,”
because researchers could not contact the NP, even with three separate attempts, to verify
eligibility. In total, 1,244 NPs from six states completed the survey. However, the response rate
was determined using three distinctive scenarios. If we assume that all of the non-responding
NPs were eligible for our study, the most conservative response rate would be 22.2%. If we
assume that approximately half of the NPs who were labeled as “unknown” in the follow-up
calls were ineligible, the response rate would be 25.7%. Lastly, if we assume that all of the non-
responding, “unknown” NPs were ineligible, the response rate is 31.8%. We anticipate the actual
response rate to be between 22.2% and 31.8%.
Current Study: We extracted data on all 396 primary care NPs practicing in either New
Jersey or Pennsylvania because these states have similar health care markets and rankings for
access to care (U.S. News & World Reports, 2019) and quality of care (U.S. News & World
Reports, 2019). As a result, NPs in these states would be practicing in similar markets. Approval
to conduct this study was obtained from the institutional review board of a large university
located in the Northeast.
75
Measures: Use of multifunctional EHRs: The EHR subscale comprised of two validated
survey items (i.e., computerized capabilities and electronic care reminder systems) which were
used to measure use of multifunctional EHRs (Friedberg et al., 2009). In previous studies
investigating use of EHRs in primary care practices, provider responses from these items (i.e.,
computerized capabilities and electronic care reminder systems) were combined to create an
overall provider-level EHR subscale score (Friedberg et al., 2009; Martsolf et al., 2018). Below
is a description of how this EHR subscale score was created.
The first item measures five computerized capabilities (i.e., recording patient history and
demographics, clinical notes, patient’s medications and allergies, ordering prescriptions, and
viewing lab or imaging results) within the EHR and NPs responded either “yes” or “no” to
whether each capability was present in their practice. The second item measures the presence of
five electronic care reminder systems for decision support of clinical guidelines (i.e.,
asthma/chronic obstructive pulmonary disease, cardiovascular disease, hypertension, congestive
heart failure, and diabetes), by asking NPs whether each reminder system is present in their
practice. Participants responded “yes” or “no” to each type of reminder system. For both of these
items, a “yes” score was coded as one and a “no” score was coded as zero. Consistent with
previous studies using these EHR items (Friedberg et al., 2009; Martsolf et al., 2018), responses
from both EHR items were combined to create a single EHR subscale. A practice-level EHR
score was calculated by averaging the individual EHR subscale responses from NPs in the same
practice. Thus, using a sample of primary care NPs from New Jersey and Pennsylvania, the EHR
subscale had a Cronbach’s alpha of 0.86.
Burnout: Burnout was measured using a non-proprietary, single item which has been
widely used across several studies (Dolan et al., 2015; Helfrich et al., 2017; Helfrich et al., 2014;
76
Edwards et al., 2018). This measure has been validated in a national sample of NPs, primary care
physicians, physician assistants, registered nurses, medical technicians, administrative clerks,
and licensed practical nurses where this non-proprietary single item burnout measure was
compared against a commonly used, standalone item from the emotional exhaustion subscale of
the Maslach Burnout Inventory (MBI:EE) (Dolan et al., 2015). The non-proprietary single item
measure is correlated with the standalone single item from the MBI:EE and has sensitivity and
specificity greater than 80% and an area under the receiver operator curve of 0.93, indicating that
this is an appropriate, less burdensome, cost-free, and reliable substitute to the standalone item in
the MBI:EE (Dolan et al., 2015). Additionally, researchers found a closely related prevalence of
burnout when using the single item extracted from the MBI:EE (36.7%) compared to the non-
proprietary single-item measure (38.5%) (Dolan et al., 2015). In this study, NPs rated their
overall level of burnout from 1 (I enjoy my work. I have no symptoms of burnout) to 5 (I feel
completely burned-out and often wonder if I can go on), with higher scores indicating greater
burnout. In accordance with previous published studies, burnout responses were dichotomized so
scores 3 to 5 were combined to indicate “burnout,” and scores 1 to 2 combined to indicate “no
burnout” (Edwards et al., 2018; Helfrich et al., 2014).
Individual NP and practice level covariates: We obtained variables measuring
characteristics of NPs including their race, age, sex, education, marital status, years employed in
current practice, number of hours worked per week, workload, and years of NP experience, and
variables measuring characteristics of the primary care practice including size, weekend and
night hours of operation, and type of primary care practice.
Statistical Analysis: Using SPSS statistical software (IBM Corp, 2017), we cleaned,
coded, and removed outlier variables from the dataset and used boxplots to check the distribution
77
of scores for continuous variables. Variance inflation factor (VIF) statistics were used to detect
the presence of multicollinearity. The absence of multicollinearity was determined when VIF
was less than five (Akinwande et al., 2015).
Multiple Imputation: Since up to 12% of the data were missing and presence of missing
data can limit the statistical power and lead to biased estimates (Kang, 2013), we conducted
multiple imputation analyses. Multiple imputation is methodologically rigorous compared to
other approaches, such as mean substitution, since each missing observation is replaced with a
set of stimulated potential values (Murray, 2018; McCleary, 2002). We conducted multiple
imputation analyses using 10 stimulated versions of the dataset which was determined using the
percentage of missing data (Bodner, 2008). From these 10 stimulated versions, we merged and
adjusted the obtained coefficients and standard errors for missing data (Murray, 2018; McCleary,
2002). This pooled, estimated dataset was extracted and imported into STATA statistical
software (StataCorp LP, 2015) for further analyses.
Multi-level Analyses: To test if use of multifunctional EHRs predicted NP burnout, we
ran a multi-level cox regression model with time as a constant variable to estimate risk ratio.
This model fits best for our analyses because the prevalence of NP burnout was greater than
10%, and when that occurs the odds ratio in logistic regression models can overestimate the
results and the suggested alternative is cox regression (Diaz-Quijano, 2012). Cox-regression
models identify the hazard ratio (Stare & Maucort-Boulch, 2016). In our study, the use of a
constant time variable allows the hazard ratio to be the same as risk ratio (Diaz-Quijano, 2012).
Therefore, we use the term “risk ratio” instead of “hazard ratio” throughout this paper. In
addition to reporting the risk ratio, we reported the 95% confidence interval and the p-values to
show the strength of relationship between use of multifunctional EHRs and burnout. The multi-
78
level cox regression model contained variables measuring characteristics of NPs within level one
of the model, including age, sex, race, education, marital status, years of NP experience,
workload, hours worked per week, and survey modality (online or mail). The outcome variable,
NP burnout, was also a level one variable since burnout is a unique feeling experienced by the
individual NP. Variables measuring use of multifunctional EHRs, main practice site, weekend
hours of operation, evening hours of operation, and practice size were practice-level variables
within level two of the model. State was included as a fixed-effects variable.
4.12 Results
Overall, 396 NPs were included in the study (Table 1). Majority of NPs were from
Pennsylvania (72.5%) and the rest were from New Jersey. The average age of NPs was 49.5
years (Standard Deviation [SD] = 12.0 years), 90.4% were female, 89.4% White, 87.6% had a
Master’s degree as their highest educational degree, and 79.5% were married. NPs were
employed in their current practice for an average of 6.4 years (SD = 6.3 years). Over 60% of NPs
reported working in a physician owned practice. Also, NPs reported that most of their workload
consisted of providing direct patient care (average = 31.1 hours [SD = 10.1 hours]), followed by
coordinating patient care (average = 5.6 hours [SD = 6.0 hours]). Nearly 25% of primary care
NPs reported burnout.
Almost all primary care practices had computerized capabilities (i.e., recording patient
history and demographics, clinical notes, patient’s medications and allergies, ordering
prescriptions, and viewing lab or imaging results). However, the presence of electronic care
reminder systems for decision support of clinical guidelines varied among practices with only
51.3% of NPs reported that there were electronic care reminder systems for patients with
congestive heart failure, 54.8% for asthma/chronic obstructive pulmonary disease, 58.1% for
79
cardiovascular disease, 59.1% for hypertension, and 65.2% for diabetes. The average practice-
level score for use of multifunctional EHRs was 0.76 (SD = 0.24).
Multicollinearity was not detected in the final model (VIF = 1.70). In the multi-level cox
regression model, a significant negative association was found between use of multifunctional
EHRs and the risk of NP burnout (Risk Ratio = 0.30, 95% CI = 0.13 to 0.71, p = .01). The use of
multifunctional EHRs was associated with 70% lower risk of NP burnout, after controlling for
provider and practice level characteristics (Table 2).
4.13 Discussion
In this study, we investigate the association between use of multifunctional EHRs and
primary care NP burnout. While we found that nearly a quarter of primary care NPs from New
Jersey and Pennsylvania are experiencing burnout, the use of multifunctional EHRs did not
increase NP burnout. Instead, it appears that use of multifunctional EHRs (computerized
capabilities and electronic care reminder systems) is associated with 70% lower risk of NP
burnout.
Our findings are contrary to findings reported by other researchers who suggest that
greater use of EHRs in the primary care setting is associated with physician burnout (Robertson
et al., 2017; Babbott et al., 2014). The differences in our findings may partly be because we
collected survey data from 2018-2019, which is recent, and so primary care NPs may be in
practices that are better equipped with using the EHR, since EHRs were recommended for
practice nearly 11 years ago (Gordon et al., 2015). Furthermore, some advantages of EHRs, such
as reliable prescribing, complete documentation that could be shared with other clinicians, and
improved coordination of health care services may assist NPs in providing patient care (Office of
the National Coordinator for Health Information Technology, 2019), thereby reducing their
80
workload and potentially reducing burnout. Since it is unknown whether these specific features
of the EHR serve as a protective mechanism against NP burnout, future research studies
examining the relationships between other components of the EHR and NP burnout are
recommended.
Although in this study, use of multifunctional EHRs is operationalized by the presence of
computerized capabilities and electronic reminder systems both of which are not associated with
increased NP burnout, there may still be other components of the EHR, which are not
investigated in this study, contributing to NP burnout. For example, we did not investigate how
much time NPs spend on the EHR and whether more time spent on the EHR is associated with
NP burnout, but researchers have found that increased time spent on the EHR contributes to
physician burnout (Arndt et al., 2017). Thus, we cannot conclude that the EHR as a whole
reduces NP burnout, but we can only conclude that two features of the EHR (i.e., computerized
capabilities and electronic reminder systems) are not associated with NP burnout. In addition,
with nearly 90% of NPs reporting that they had EHRs with computerized capabilities, therefore,
our results are more reflective of the impact of electronic reminder systems. Our results might
suggest that NPs using more reminder systems, which assist in decision support of clinical
guidelines, experience lower levels of burnout and this finding should be further explored.
Moreover, we found that personal characteristics of NPs such as their age, race, sex,
education, and marital status are not associated with burnout. Our findings are consistent with
researchers who report that the fundamental causes of burnout are not personal characteristics of
the individual but instead lie within the organization (Moss, 2019; Harrison et al., 2017; Sillero
& Zabalegui, 2018; Stearns & Benight, 2016; Dillon et al., 2020). For example, these researchers
have suggested that unmanageable and increasing workloads associated with the EHR, unfair
81
treatment of colleagues, lack of role clarity, unsupportive supervisors, and a workplace culture
focused on productivity are just some features within the organization that can lead to burnout
(Moss, 2019; Sillero & Zabalegui, 2018; Dillon et al., 2020). Therefore, future interventions
seeking to improve organizational working conditions (e.g., reducing high workloads stemming
from the EHR, improving team communication, working with supportive colleagues) may be
more fruitful in reducing burnout compared to individual, personal interventions (e.g., yoga or
meditation); however more research is needed to verify which interventions are best.
Limitations: The use of cross-sectional survey data limits our ability to determine a
causal relationship between use of multifunctional EHRs and NP burnout. Additionally, one
EHR variable (computerized capabilities) in this study had limited variability. With most NPs
reporting that their practice had computerized capabilities, it is possible that the computerized
capability items were features required to be in the EHR. Another limitation within this study is
the absence of important EHR covariates such as NP training and experience with the EHR,
availability of support with the EHR, and usability and functionality of the EHR which were not
present in the parent study. Researchers have found that these variables can influence provider
satisfaction with the EHR (Yen & Bakken, 2012), thus omitting them may bias our results. The
generalizability of our findings should be used with caution since our sample originated only in
New Jersey and Pennsylvania, therefore our results are not applicable to NPs in other states or
countries.
The survey response rate is low. While one may speculate whether the NPs in our survey
are different from the overall NP population, we found that NPs in our study are relatively
similar in race (89.4% White vs. 87% White nationally) and sex (90.4% female vs. 91.7% female
nationally) compared to NPs from the National Nurse Practitioner Sample Survey (AANP,
82
2019). Despite the limitations, our results may provide new information on primary care NP use
of multifunctional EHRs and their burnout. Knowing how primary care NPs are using the EHR
can help in the development of future studies examining what features of the EHR help NPs to
deliver coordinated primary care to patients.
4.14 Conclusion
To our knowledge, this is one of the first studies investigating primary care NPs use of
multifunctional EHRs and their experiences of burnout. Although 25% of NPs reported
experiencing burnout, the use of multifunctional EHRs was not associated with NP burnout.
While our findings are contrary to prior studies on EHR use and physician burnout, it may be
that the NPs in our study became more adjusted and familiar with operating EHRs since the
passage of the Health Information Technology Act in 2009. Nonetheless, additional research
studies with robust EHR specific covariates are needed to further examine the usability and
operability of the EHR among all PCPs.
83
4.15 Figure 1
Adapted Clinician Well-Being and Resilience Model
Note. EHRs = Electronic Health Records;
84
4.16 Table 1
Characteristics of Primary Care Nurse Practitioners (N = 396)
N (%) Mean (SD) Age 49.5 years (12.0) Female 358 (90.4) Highest Educational Degree Master’s degree 347 (87.6) Doctorate of Nursing Practice (DNP) 28 (7.1) PhD or other doctorate 12 (3.0) Other 9 (2.3) Race White 354 (89.4) Black of African American 11 (2.8) American Indian or Alaska Native 2 (0.5) Asian 18 (4.5) Native Hawaiian or other Pacific islander 5 (1.3) Other 6 (1.5) Hispanic/Latino 12 (3.0) Marital status- Married 315 (79.5) Years of NP experience 11.4 years (8.9) Years employed in current practice 6.4 years (6.3) Average number of hours worked per week 38.9 hours (11.2) NP workload Providing direct patient care 31.1 hours (10.1) Coordinating patient care 5.6 hours (6.0) Providing care management services 2.0 hours (3.3)
Performing quality assurance and improvement activities
1.6 hours (3.5)
Conducting administrative activities/leadership 1.7 hours (3.9) Survey Modality Mail 309 (78.0) Online 87 (22.0) Burnout 100 (25.3) Characteristics of Primary Care Practices Practice Location New Jersey 109 (27.5) Pennsylvania 287 (72.5) Main practice site Physician practice 239 (60.4) Community health center 51 (12.9) Hospital based clinic 37 (9.3) Retail-based clinic 8 (2.0) Urgent care clinic 12 (3.0)
85
Nurse managed clinic 4 (1.0) Other 45 (11.4) Practice Open on Weekends 141 (35.6) Practice Open at Night None 127 (32.1) Once a week 78 (19.7) Twice a week 82 (20.7) Three times a week 24 (6.1) Four or more times a week 85 (21.5) Computerized capabilities present within the practice Recording patient history 388 (98.0) Recording clinical notes 385 (97.2) Recording patient’s medications and allergies 387 (97.7) Ordering prescriptions 380 (96.0) Viewing lab or imaging results 379 (95.7) Presence of electronic reminder systems in the practice for patients with: Asthma/COPD 217 (54.8) Cardiovascular Disease 230 (58.1) Hypertension 234 (59.1) Congestive Heart Failure 203 (51.3) Diabetes 258 (65.2) Use of Multifunctional EHRs 0.76 (0.24)
Note. SD = Standard Deviation; NP = Nurse Practitioners; COPD = chronic obstructive pulmonary disease;
86
4.17 Table 2
Multi-level Cox Regression model, with time as a constant variable, assessing the effect of use of
multifunctional EHRs on NP burnout while controlling for provider and practice level covariates
(n = 396)
Risk Ratio*
P > |z| 95% Confidence Interval
Use of multifunctional EHRs 0.3 0.01 0.13 0.71 Average hours worked/week 1 0.82 0.98 1.03 Age 0.99 0.41 0.97 1.01 Female 1.21 0.61 0.58 2.51 Marital Status- Married 0.8 0.34 0.50 1.27 Education Master’s Degree 3.15 0.27 0.42 23.92 Doctorate of Nursing Practice 2.83 0.34 0.34 23.98 Other Degree 3.04 0.38 0.26 35.8 Race- White 1.31 0.49 0.61 2.83 Years of Experience as an NP 1.01 0.65 0.98 1.04 Workload^ Providing patient care 1.01 0.3 0.99 1.04 Coordinating patient care 1.01 0.66 0.97 1.05 Providing care management services
0.99 0.81 0.92 1.07
Performing quality improvement 1.02 0.61 0.94 1.1 Practice leadership and administrative tasks
1 0.97 0.94 1.07
Type of practice - Physician practice 0.91 0.65 0.60 1.38 Open during the Weekends 0.89 0.66 0.54 1.47 Open at Night 0.94 0.41 0.82 1.08 Practice Size 1 0.45 0.99 1.03 Survey type 1.11 0.68 0.69 1.77 State 0.8 0.33 0.50 1.26
Note. NP = Nurse Practitioner; EHRs = Electronic Health Records; Workload^ = Measured as hours per week performing the following tasks; * = Since we did not have NP responses recorded over various time points, we created a constant time variable. With time as a constant variable, the hazard ratio is the same as the risk ratio. Thus, we reported the risk ratio in this table.
87
Chapter 5: Primary care Nurse Practitioner Burnout and Quality of
Care
In this chapter, the author investigates the relationship between primary care NP burnout
and quality of care and if the practice environment moderates the relationship between burnout
and quality of care. This manuscript has been submitted to a peer reviewed journal.
88
5.1 Introduction
Clinician burnout is identified by feelings of emotional exhaustion, depersonalization,
and low personal accomplishment (Maslach & Jackson, 1981). Over the last two decades,
researchers have found that burnout is prevalent among clinicians and consequently impacts
quality of care and patient outcomes within the acute care setting (West et al., 2018; Panagioti et
al., 2018). In a recent study, 44% of cardiologists and 50% of neurologist experience burnout
(Kane, 2020). The high prevalence of burnout is concerning because clinician burnout negatively
affects both clinicians and patients (West et al., 2018). Burnt-out clinicians are more likely to
report job stress and dissatisfaction, fatigue, irritability, and abuse alcohol and drugs (Rabatin et
al., 2016; Abdulla et al., 2011; Soler et al., 2008). Furthermore, clinician burnout negatively
affects patients as it is associated with urinary tract and surgical site infections (Cimiotti et al.,
2012), low patient satisfaction, poor quality of care (Nantsupawat et al., 2016; Panagioti et al.,
2018; Dewa et al., 2017) and high rates of medical errors thereby, compromising patient safety
(West et al., 2018).
Despite the adverse clinician and patient outcomes associated with burnout, most studies
that currently exist are focused on clinicians in the acute care settings (Tourigny et al., 2019;
Wen et al., 2016; Van Bogaert et al., 2013; Cimiotti et al., 2012). This is a limitation because
most health care delivery occurs in primary care settings provided by primary care providers
(PCPs) who are physicians, physician assistants, and nurse practitioners (NPs) (Petterson et al.,
2018). Studies conducted in the acute care setting cannot be generalized to primary care. Of the
studies investigating burnout among PCPs, most explore primary care physician burnout
(Robertson et al., 2017; Rabatin et al., 2016; Babbott et al., 2014; Linzer et al., 2009). Some of
these researchers found that 37% of primary care physicians were burnt-out (Robertson et al.,
89
2017), which is concerning since burnt-out physicians are also more likely to deliver suboptimal
patient care (Dewa et al., 2017). However, no study author, to date, has reported the prevalence
and outcomes associated with primary care NP burnout which is a limitation because nearly a
third of PCPs are projected to be NPs by 2030 (U.S. Department of Health and Human Services,
2016) and close to 70% of NPs currently deliver primary care services to patients (AANP, 2020).
Currently, a scarcity of research exists on the association between primary care NP
burnout and the practice environment. Poor practice environments are a major contributor to
burnout in the primary care setting (Abraham et al., 2019), and an extensive body of evidence
exists on the challenges within the practice environment that are unique to NPs (Poghosyan et
al., 2013; Poghosyan & Aiken, 2015; Poghosyan et al., 2017). For example, NPs in poor practice
environments have inadequate support and resources from administrators, suboptimal
communication with staff, limited autonomy (Poghosyan et al., 2013), and such organizational
challenges may predispose NPs to burnout and impact the quality of care they deliver to patients.
In another study, 46.1% of NPs were working in a poor practice environment (Carthon et al.,
2020) which is concerning because numerous researchers have shown that working in poor
practice environments can lead to burnout (Rabatin et al., 2016; Lake et al., 2019; Aiken et al.,
2012) and poor patient outcomes (Lake et al., 2019; Lake et al., 2016; Olds et al., 2017).
However, NPs who worked in a good environment were more likely to incorporate patient
preferences into care delivery compared to NPs in either mixed or poor environments (Carthon et
al., 2020). Since the environment where clinicians deliver care has long been shown to affect
provider outcomes and patient care (Rabatin et al., 2016; Lake et al., 2019; Olds et al., 2017;
Aiken et al., 2012), it is plausible that working in a poor practice environment may lead to NP
90
burnout and the delivery of low quality patient care, however, these implications require further
investigation.
Burnout in primary care NPs can be a barrier to achieving favorable clinical outcomes
and high-quality care. Hence, identifying the prevalence of burnout in primary care NPs and its
clinical consequence for patients will be the first step in designing future interventions to
improve primary care delivery and potentially reduce NP burnout. The overarching purpose of
this study is to determine whether primary care NP burnout is associated with lower quality of
care in primary care practices and whether the NP practice environment moderates the
relationship between NP burnout and quality of care.
5.11 Methods
Conceptual Framework: Our study was informed by the Clinician Well-Being and
Resilience model (Brigham et al., 2018) (Figure 1), which exemplifies the relationship between
factors contributing to clinician burnout and outcomes of burnout such as compromised clinician
well-being, clinician-patient relationships, and patient well-being (Brigham et al., 2018). We
adapted the conceptual model to display the relationship between NP demographics, practice
characteristics, burnout, and quality of care. Burnout is conceptualized based on the NP’s own
reports of burnout which can range from NPs reporting no symptoms of burnout to completely
burned-out. Quality of care is conceptualized as an NP’s perception of the quality of care
delivered within their primary care practice. Variables measuring characteristics of NPs include
age, sex, education, marital status, race, hours worked per week, experience, years employed in
the current practice, workload, as well as practice characteristics such as size, type of practice,
and hours of operation are also included in the adapted conceptual model.
Study Design and Data Source: This current study is a secondary analysis of cross-
91
sectional survey data that were obtained from the parent study (Poghosyan et al., 2020), which
investigated ways to reduce health disparities in outcomes for chronically ill elderly adults
receiving primary care services from NPs.
Description of the Parent study: Participants & Setting: Primary care NPs were
recruited from the SK&A OneKey primary care practice database which is a comprehensive
database containing information on health care providers (e.g., NPs) in the U.S. (DesRoches et
al., 2015). Three different types of primary care practices employing NPs were included: 1) an
independent practice with a physician who has a primary care specialization and no less than one
NP employed in that practice, 2) a medical group practice that delivers primary care and employs
no less than one NP in the practice, or 3) a medical group practice with one or more physician,
at-least half of the physicians have a primary care specialization, and no less than one NP
employed in that practice. As there were more NPs in geographically larger states (i.e.,
California, Pennsylvania, Texas) than in smaller states (i.e., New Jersey, Arizona, Washington),
a random sampling of NPs was conducted.
Data Collection: From 2018-2019, primary care NPs completed surveys containing valid
measures of burnout, quality of care, practice environment, demographics, and characteristics of
their main practice site. Researchers from the parent study collected survey data using a modified
Dillman approach for mixed-mode surveys (Dillman et al., 2014), in which the mailed surveys
contained an online link to let NPs complete the surveys either on paper or electronically.
Overall, there were three rounds of mailed surveys sent to NPs and two rounds of reminder
postcards which were sent two weeks after the first and second survey mailings. To maximize
the overall response rate, follow-up phone calls to non-responding NPs were also conducted.
During some of these calls, researchers could not confirm whether the NPs were truly eligible
92
(e.g., actually worked in the primary care practice) and so these NPs were labeled as “unknown,”
non-responding NPs. In total, 1,244 NPs across six states completed the survey. The most
conservative response rate, 22.2%, was one in which we assumed that all non-responding NPs
were eligible. However, we also calculated a response rate, 31.8%, based on a scenario where all
non-responding NPs were ineligible. Therefore, we anticipate the real response rate to be in a
range from 22.2% to 31.8%.
Current Study: Our final sample contained 396 primary care NPs practicing in either
New Jersey or Pennsylvania. These two states were selected because they have comparable
rankings for access to care (U.S. News & World Reports, 2019) and quality of care (U.S. News
& World Reports, 2019); hence, NPs working in these states would be practicing in related
health care markets. We obtained approval from the institutional review board of a large
university located in the Northeast prior to conducting this study.
Measures: Burnout: Primary care NP burnout was measured using a non-proprietary,
single-item measure. This single-item was validated in a nationally representative sample of
primary care workers (i.e., physicians, NPs, physician assistants, registered nurses, medical
technicians, administrative clerks, and licensed practical nurses) by comparing this single item
burnout measure against a standalone single item from the emotional exhaustion subscale of the
Maslach Burnout Inventory (Dolan et al., 2015). Dolan and colleagues (2015) concluded that the
prevalence of burnout was relatively similar while using the non-proprietary single item measure
(38.5%) rather than the single item from the emotional exhaustion subscale of Maslach Burnout
Inventory (36.7%). For this present study, NPs self-reported their experience of burnout from
one (I enjoy my work. I have no symptoms of burnout) to five (I feel completely burned-out and
often wonder if I can go on). Higher scores suggested greater burnout (Dolan et al., 2015).
93
Consistent with prior studies investigating burnout among PCPs (Edwards et al., 2018; Helfrich
et al., 2014), burnout responses were dichotomized by combining scores three to five to indicate
“burnout,” and scores one to two to indicate “no burnout.”
Practice Environment: The practice environment was measured using the Nurse
Practitioner Primary Care Organizational Climate Questionnaire (NP-PCOCQ), which is a 29-
item survey containing the following four subscales: Professional Visibility (PV) (4 items), NP-
Physician Relations (NP-PR) (7 items), NP-Administration Relations (NP-AR) (9 items), and
Independent Practice and Support (IPS) (9 items) (Poghosyan et al., 2013). The NP-PR subscale
measured the level of collegiality and teamwork between physicians and NPs, the PV subscale
measured how well NP role is understood within the organization, the NP-AR subscale measured
NP perceptions of their relations with organization administrators, and the IPS subscale
measured the presence of organizational support and resources available to NPs (Poghosyan et
al., 2013). Since each NP-PCOCQ subscale represents a different dimension of the practice
environment, we did not combine scores across the four subscales to generate one practice
environment score. Using a sample of 396 primary care NPs from New Jersey and Pennsylvania,
the Cronbach’s alpha for the PV subscale was 0.85, the NP-AR subscale was 0.94, the IPS
subscale was 0.87, and the NP-PR subscale was 0.88.
Responses for each NP-PCOCQ survey item ranged from “strongly agree” to “strongly
disagree” (Poghosyan et al., 2013). Scores for the individual NP-PCOCQ subscales ranged from
one to four, with higher mean subscale scores indicating better NP practice environments.
Practice-level scores for each NP-PCOCQ subscale were generated by aggregating the individual
mean subscale scores of NPs in the same practice. To avoid the possibility of high
94
multicollinearity, each practice-level NP-PCOCQ subscale score was centered to have a mean of
zero before being included in the moderation analysis (Shieh, 2011; Aiken & West, 1991).
Quality of care: Quality of care was measured by asking NPs “how would you rate the
quality of care your organization delivers as a whole?” Response options are on a 5-point Likert
scale, ranging from “poor” to “excellent,” with higher scores indicating better perceptions of care
quality. Researchers have found that nurse-perceived quality of care is a consistent indicator of
the quality of patient care in outcomes such as failure to rescue, mortality, and patient
satisfaction (McHugh & Stimpfel, 2012). Furthermore, various researchers have used nurse-
perceived quality of care as an outcome measure and have produced valuable, clinically
significant results (Ball et al., 2017; Stalpers et al., 2016; Nantsupawat et al., 2016). Thus, NP
perceptions of quality of care may provide valuable information on the quality of care delivered
to patients within their organizations.
Provider and Practice-level covariates: We attained variables measuring the demographic
characteristics of NPs (i.e., age, sex, race, marital status, highest educational degree, years
employed in current practice, number of hours worked per week, workload, and years of NP
experience) and variables measuring practice characteristics (i.e., size, hours of operation, and
the type of primary care practice).
Statistical Analysis: The dataset was cleaned and variables were coded in SPSS (IBM
Corp, 2017). The distribution of scores for continuous variables and potential outlier scores were
identified using boxplots within SPSS. Outlier values were removed from the dataset. To
determine if multicollinearity was present, we used variance inflation factor (VIF) statistics. A
VIF value less than five suggested no multicollinearity (Akinwande et al., 2015).
95
Multiple Imputation: We used multiple imputation analyses because some variables in
our study had up to 12% of missing data and multiple imputation allows researchers to obtain
accurate results, prevent a potential loss in statistical power, and generate valid variance
estimates when there is missing data (Murray, 2018; Stuart et al., 2009; McCleary, 2002). Given
these benefits, we conducted multiple imputation analyses by creating 10 simulated versions of
the dataset which was ascertained from the percentage of missing data (Bodner, 2008). The
results from the 10 complete datasets were combined and adjusted for obtained coefficients and
standard errors for missing data (Murray, 2018; McCleary, 2002). All multi-level analyses were
conducted using the pooled estimates from the multiple imputation analyses.
We extracted and imported the pooled data into STATA statistical software (StataCorp
LP, 2015) to conduct all multi-level analyses. For Aim 1, to determine if NP burnout is
associated with quality of care, we conducted a multi-level proportional odds cumulative logit
model containing variables measuring burnout, quality of care, and characteristics of NPs within
level one of the model and variables measuring characteristics of the practice within level two of
the model. Since NPs had the option of completing either mail or online surveys, we controlled
for survey modality within our multi-level models. State was included as a fixed effects variable.
Moderation analysis: For Aim 2, a Baron and Kenny moderation analysis (Baron &
Kenny, 1986) was conducted to assess if each NP-PCOCQ subscale moderates the relationship
between burnout and quality of care. Figure 2 illustrates a path diagram representing the
moderation model (Baron & Kenny, 1986). As shown in Figure 2, there are three paths that can
influence the outcome variable which is quality of care. Path A tests the impact of NP burnout on
quality of care. Path B tests the impact of the moderator variable (i.e., the four practice
96
environment subscales) on quality of care. Lastly, path C tests the impact of the interaction term
on quality of care.
To test for moderation, we ran two separate models for each of the practice-level PV,
IPS, NP-PR, and NP-AR subscales after centering these subscale scores to a mean of zero. In the
first multi-level proportional odds cumulative logit model, quality of care was the outcome
variable, and the two independent variables were burnout and the centered practice-level practice
environment subscale (i.e., either PV, IPS, NP-PR, or NP-AR). In the second model, the only
newly added variable was the centered interaction term which was created by multiplying
burnout with the centered practice-level practice environment subscale. A cross-level interaction
term was created because we had variables within level one (NP level) and level two (practice
level) of the multi-level model. If the interaction variable was statistically significant, it was
reported that the practice environment subscale moderates the relationship between burnout and
quality of care. Among all models, significance values less than 0.05 were deemed statistically
significant.
5.12 Results
There were 396 NPs included in the study with nearly 73% of NPs from Pennsylvania
(Table 1). Most NPs were female (90.4%), White (89.4%), had a Master’s degree (87.6%), and
had an average of 11.4 years of NP experience (Standard Deviation [SD] = 8.9 years). None of
the NPs perceived delivering poor quality of care, 4.3% perceived fair, 14.9% perceived good,
43.4% perceived very good, and 37.4% perceived delivering excellent quality of care. Nearly
25% of NPs reported burnout. No multicollinearity was detected in the final model, since the
VIF was 1.69. In the multi-level model, NP burnout was associated with quality of care. The
odds of perceiving higher quality of care was 85% less for NPs experiencing burnout, as
97
compared to NPs not experiencing burnout (Cumulative Odds Ratio [COR] = 0.15, 95% CI =
0.09 to 0.24, p = .000), after controlling for NP and practice level covariates (Table 2).
Moderation Analysis: To test the hypothesis if the NP practice environment moderates
the relationship between NP burnout and quality of care, two multi-level proportional-odds
cumulative logit models were conducted for each practice environment subscale (Tables 3 and
4).
Professional Visibility: In the first model, burnout and the mean PV subscale score were
both significantly associated with quality of care (p < .00). With a one unit increase in the mean
PV subscale, the odds of perceiving higher quality of care increased to 4.39 times (COR = 4.39,
95% CI = 3.10 to 6.20, p = .000). In the second model, the cross-level interaction variable
between burnout and the mean PV subscale score was not significant (Beta Coefficient = -0.10, p
= .77).
Independent Practice and Support: Similar to the PV subscale, the cross-level interaction
variable between burnout and the mean IPS subscale score was not significant (Beta Coefficient
= -0.005, p = .99). However, without the interaction term, burnout and the mean IPS subscale
score were significantly associated with quality of care (p < .00). With a one unit increase in the
mean IPS subscale, the odds of perceiving higher quality of care increased to 7.57 times (COR =
7.57, 95% CI = 4.77 to 12.03, p = .000).
NP-Physician Relations: In the multi-level proportional-odds cumulative logit model, the
cross-level interaction variable between burnout and the mean NP-PR subscale score was not
significant (Beta Coefficient = -0.20, p = .63). When the interaction term is removed, with a one
unit increase in the mean NP-PR subscale, the odds of perceiving higher quality of care increased
to 4.76 times (COR = 4.76, 95% CI = 3.15 to 7.20, p = .000).
98
NP-Administration Relations: The cross-level interaction variable between burnout and
the mean NP-AR subscale score was not significant (Beta Coefficient = -0.27, p = .43).
Nevertheless, with a one unit increase in the mean NP-AR subscale, the odds of perceiving
higher quality of care increased to 3.83 times (COR = 3.83, 95% CI = 2.76 to 5.30, p = .000).
5.13 Discussion
In this study, we examine whether NP burnout is associated with quality of care in
primary care practices and whether NP practice environment moderates the relationship between
burnout and quality of care. We found that nearly 25% of NPs report burnout and that an
association exists between NP burnout and lower perceptions of care quality, since the odds of
NPs perceiving higher quality of care was 85% less for NPs experiencing burnout, as compared
to NPs not experiencing burnout. Furthermore, we found that the primary care NP practice
environment did not moderate the relationship between NP burnout and quality of care. This
finding is important because it suggests that, specifically within our sample of primary care NPs,
the practice environment does not explain the link between NP burnout and quality of care.
Although there is no moderating effect, we did find that the primary care practice environment is
independently associated with quality of care. In particular, with a one unit increase in the mean
PV, IPS, NP-PR, and NP-AR subscales, the odds of perceiving higher quality of care increased
to 4.39, 7.57, 4.76, and 3.83 times, respectively. This implies that working environments where
administrators and staff members have a good understanding of the NP role, NPs are allowed to
provide care within their scope of practice, and NPs have positive relations with physicians and
practice administrators all contribute to NPs’ perceptions of delivering higher quality of care. As
such, fostering a healthy practice environment for an NP may be an important mechanism to
improving patient care delivery.
99
Our findings are consistent with researchers who also report that nurse burnout is
associated with lower perceptions of care quality (Nantsupawat et al., 2016; Poghosyan et al.,
2010). One researcher found that for every unit increase in the emotional exhaustion score for
nurses, there was a 2.53 times increase in reporting fair or poor quality of patient care
(Nantsupawat et al., 2016), which shows that burnout in nurses can influence nurse perceptions
of care quality. Moreover, our findings are also consistent with the larger body of evidence
showing that when nurses practice in favorable environments, perceptions of care quality may
also improve (Lake et al., 2016; Lake et al., 2019; Kanai-Pak et al., 2008). For example, in better
practice environments, nurses were 66% less likely to perceive fair or poor quality of care
compared to nurses in poor environments (Lake et al., 2016). As a result, one may infer that
improvements made to the NP practice environment (e.g., organizational support for NPs,
adequate supply of resources to provide care, optimal relationships with physicians and
administrators, and professional visibility within their practice) may help to subsequently
improve NP perceptions of care quality. In a systematic review on interventions to reduce
physician burnout, researchers report that improving the practice environment by reducing
workload and time pressure, improving team communication and physician involvement in
organizational changes, and working with a supportive supervisor can reduce burnout
(Wiederhold et al., 2018). Therefore, it may be possible that such improvements made to the NP
practice environment may also reduce NP burnout.
Implications for Clinical Practice and Future Research: Clinicians and practice
administrators can view the independent associations between the practice environment and
quality of care as evidence showing how working in favorable practice environments is
associated with perceptions of delivering higher quality of patient care. As a result,
100
organizational changes designed to improve the practice environment and reduce NP burnout
may also improve quality of patient care. To ensure that patients receive the highest quality of
care in primary care practices, future research should involve all PCPs to see how the practice
environment for NPs, physicians, and physician assistants can be modified to reduce provider
burnout and optimize care quality for patients. Given the high prevalence of NP burnout, it is
also recommended that researchers conduct studies to investigate what interventions are best to
reduce primary care NP burnout.
Limitations: Since this study uses cross-sectional survey data, it is not possible to
determine whether burnout causes lower NP-perceive quality of care. Additionally, the data
collected in the parent study uses NP perceptions of quality of care which may be different from
patient reported quality of care. However, researchers have successfully measured perceptions of
care quality among primary care physicians (Pantell et al., 2019), and provider perceptions of
quality of care has been shown to be a reliable indicator for quality of care delivered to patients
(McHugh & Stimpfel, 2012). The overall response rate is low which may suggest that the NPs
who completed the survey are different from NPs nationally. However, NPs in our study were
fairly akin in race (89.4% White vs. 87% White nationally), age (49.5 years vs. 49 years
nationally), and sex (90.4% female vs. 91.7% female nationally) to NPs nationally (AANP,
2019). Moreover, since this study investigates burnout among primary care NPs in New Jersey
and Pennsylvania, the generalizability of our findings is limited and may not be applicable to
NPs in other states and countries. Nonetheless, findings from our study may be of interest to
clinicians and administrators seeking to improve the quality of care delivered within their
organizations.
101
5.14 Conclusion
This study examined the relationship between primary care NP burnout and quality of
care in practices located in New Jersey and Pennsylvania. We found that NP burnout was
associated with lower perceptions of delivering higher quality of care. Similarly, favorable
primary care NP practice environments were associated with higher perceptions of care quality,
but the practice environment did not moderate the relationship between burnout and quality of
care. Ultimately, results from our study provide valuable insight for practice administrators to
prioritize reducing primary care NP burnout and to create favorable practice environments for
NPs, which may contribute to the delivery of high quality patient care.
102
5.15 Figure 1
Adapted Clinician Well-Being and Resilience model
103
5.16 Figure 2
Adapted Moderation Model by Baron and Kenny (1986)
Note. NP = Nurse Practitioner; Each practice environment subscale has a separate moderation model.
104
5.17 Table 1
Characteristics of Primary Care Nurse Practitioners (N = 396)
N (%) Mean (SD) Age 49.5 years (12.0) Female 358 (90.4) Highest Educational Degree Master’s degree 347 (87.6) Doctorate of Nursing Practice (DNP) 28 (7.1) PhD or other doctorate 12 (3.0) Other 9 (2.3) Race White 354 (89.4) Black of African American 11 (2.8) American Indian or Alaska Native 2 (0.5) Asian 18 (4.5) Native Hawaiian or other Pacific islander 5 (1.3) Other 6 (1.5) Hispanic/Latino 12 (3.0) Marital status Never Married 27 (6.8) Married 315 (79.5) Separated 5 (1.3) Divorced 39 (9.8) Widowed 10 (2.5) Years of NP experience 11.4 years (8.9) Years employed in current practice 6.4 years (6.3) Average number of hours worked per week 38.9 hours (11.2) NP workload Providing direct patient care 31.1 hours (10.1) Coordinating patient care 5.6 hours (6.0) Providing care management services 2.0 hours (3.3)
Performing quality assurance and improvement activities
1.6 hours (3.5)
Conducting administrative activities/leadership 1.7 hours (3.9) Survey Modality Mail 309 (78.0) Online 87 (22.0) Burnout 100 (25.3) Quality of Care Poor 0 (0.0) Fair 17 (4.3) Good 59 (14.9)
105
Very good 172 (43.4) Excellent 148 (37.4) Characteristics of Primary Care Practices Practice Location New Jersey 109 (27.5) Pennsylvania 287 (72.5) Main practice site Physician practice 239 (60.4) Community health center 51 (12.9) Hospital based clinic 37 (9.3) Retail-based clinic 8 (2.0) Urgent care clinic 12 (3.0) Nurse managed clinic 4 (1.0) Other 45 (11.4) Practice Open on Weekends 141 (35.6) Practice Open at Night None 127 (32.1) Once a week 78 (19.7) Twice a week 82 (20.7) Three times a week 24 (6.1) Four or more times a week 85 (21.5) Primary Care NP Practice Environment Subscale Scores Professional Visibility 3.11 (0.65) NP-Administration Relations 2.82 (0.71) NP-Physician Relations 3.33 (0.52) Independent Practice and Support 3.42 (0.48)
Note. SD = Standard Deviation; NP = Nurse Practitioners; Higher mean scores on the practice environment subscale indicate more favorable practice environments for NPs; Higher scores on the quality of care scale indicates the PCP’s report of delivering the best possible care to patients in that organization.
106
5.18 Table 2
Multi-level proportional-odds cumulative logit models assessing the effect of burnout on quality
of care while controlling for provider and practice level covariates (n = 396)
Cumulative Odds Ratio
P > |z| 95% Confidence Interval
Burnout 0.15 0.00 0.09 0.24 Average hours worked/week 0.99 0.61 0.97 1.02 Age 1.00 0.89 0.98 1.02 Female 0.96 0.90 0.48 1.91 Marital Status- Married 0.78 0.31 0.48 1.27 Education Master’s Degree 1.12 0.86 0.33 3.80 Doctorate of Nursing Practice 1.64 0.49 0.41 6.56 Other Degree 0.66 0.65 0.11 3.95 Race- White 1.21 0.58 0.62 2.35 Years of Experience as an NP 1.02 0.13 0.99 1.05 Workload^ Providing patient care 1.02 0.12 0.99 1.05 Coordinating patient care 0.98 0.27 0.94 1.02 Providing care management services
1.02 0.70 0.94 1.10
Performing quality improvement 1.07 0.10 0.99 1.16 Practice leadership and administrative tasks
1.00 0.92 0.94 1.06
Type of practice - Physician practice 1.28 0.24 0.85 1.93 Open during the Weekends 1.08 0.74 0.68 1.73 Open at Night 1.09 0.20 0.96 1.24 Practice Size 1.03 0.05 1.00 1.06 Survey type 0.88 0.60 0.55 1.42 State 0.46 0.00 0.29 0.74
Note. NP = Nurse Practitioner; Workload^ = Measured as hours per week performing the following tasks;
107
5.19 Table 3
Multi-level proportional-odds cumulative logit models assessing the effect of burnout and
practice environment with quality of care (n = 396)
Cumulative Odds Ratio
P > |z| 95% Confidence Interval
Model 1 for Professional Visibility Subscale = Burnout 0.23 0.00 0.14 0.36 Professional Visibility 4.39 0.00 3.10 6.20
Model 1 for Independent Practice and Support Subscale =
Burnout 0.21 0.00 0.13 0.34 Independent Practice and Support 7.57 0.00 4.77 12.03
Model 1 for NP-Physician Relations Subscale =
Burnout 0.21 0.00 0.13 0.34 NP-Physician Relations 4.76 0.00 3.15 7.20 Model 1 for NP-Administration Relations Subscale = Burnout 0.27 0.00 0.16 0.43 NP-Administration Relations 3.83 0.00 2.76 5.30
Note. NP = Nurse Practitioner;
108
5.20 Table 4
Multi-level proportional-odds cumulative logit models testing whether the practice environment
moderates the relationship between burnout and quality of care (n = 396)
Quality of Care Beta Coefficient
P > |z| 95% Confidence Interval
Model 2 for Professional Visibility Subscale = Burnout -1.51 0.00 -2.00 -1.02 Professional Visibility 1.51 0.00 1.10 1.93 Interaction Term (Burnout*Professional Visibility)
-0.10 0.77 -0.80 0.59
Model 2 for Independent Practice and Support Subscale =
Burnout -1.54 0.00 -2.03 -1.06 Independent Practice and Support 2.03 0.00 1.48 2.57 Interaction Term (Burnout*Independent Practice and Support)
-0.005 0.99 -0.92 0.91
Model 2 for NP-Physician Relations Subscale =
Burnout -1.58 0.00 -2.07 -1.10 NP-Physician Relations 1.63 0.00 1.12 2.14 Interaction Term (Burnout*NP-Physician Relations)
-0.20 0.63 -1.02 0.62
Model 2 for NP-Administration Relations Subscale =
Burnout -1.39 0.00 -1.90 -0.88 NP-Administration Relations 1.42 0.00 1.04 1.81 Interaction Term (Burnout*NP-Administration Relations)
-0.27 0.43 -0.93 0.40
Note. NP = Nurse Practitioner;
109
Chapter 6: Recommendations for Research, Practice, and Policy
In this chapter, the author 1) discusses the results of all four studies encompassing this
dissertation, 2) describes the strengths and limitations of the dissertation, and 3) provides
recommendations for research, practice, and policy.
110
6.1 Discussion of Findings
This dissertation encompasses one systematic review investigating predictors and
outcomes of burnout among all primary care providers (PCPs) and three additional cross-
sectional studies which investigated factors associated with primary care Nurse Practitioner (NP)
burnout. In study one of this dissertation, we identified the predictors and outcomes of burnout
among all PCPs in the United States. In this systematic review consisting of 21 studies, we found
that high workload, poor primary care practice environments, and use of multifunctional EHRs
were commonly reported predictors of PCP burnout (Abraham et al., 2019). Since the studies
included in the systematic review mostly investigated primary care physician burnout, there
remained a gap in the literature on factors associated with primary care NP burnout. As a result,
the remaining studies in this dissertation were conducted to fill this gap by specifically
investigating factors associated with primary care NP burnout.
In study two of this dissertation, we examined whether the primary care practice
environment was associated with NP burnout. The aim of this study was achieved by using
cross-sectional survey data collected from 396 NPs from New Jersey and Pennsylvania to
conduct multi-level cox regression models, that included NP demographics, burnout, and survey
modality in level one of the model and variables measuring the practice environment, and
characteristics of the primary care practice in level two of the model. Multi-level models are
ideal to use because they can account for the hierarchical effect of provider and practice level
variables (Nezlek, 2008). The overall prevalence of burnout among 396 primary care NPs from
New Jersey and Pennsylvania was 25.3%. In the multi-level models, we found that higher mean
scores on the Professional Visibility, NP-Physician Relations, NP-Administration Relations, and
Independent Practice and Support subscales, of the tool measuring NP practice environment,
111
were associated with 51%, 51%, 58% and 56% lower risk of NP burnout, respectively. We
conclude that nearly a quarter of primary care NPs from New Jersey and Pennsylvania are burnt-
out, and the primary care practice environment is associated with NP burnout.
For study three, we tested the association between the use of multifunctional EHRs and
primary care NP burnout. Using data from the same sample of 396 NPs and similar methods as
in study two, we ran a multi-level cox regression model and found that the use of multifunctional
EHRs was associated with 70% lower risk of NP burnout. Although this finding was contrary to
our hypothesis, it may be because use of multifunctional EHRs was operationalized by the
presence of only two features: computerized capabilities and electronic reminder systems for
decision support of clinical guidelines. Hence, we can only conclude that these two features of
the EHR did not contribute to primary care NP burnout.
Lastly, in study four, we investigated whether NP burnout was associated with lower NP
perceptions of care quality in primary care practices. We also tested if each primary care practice
environment subscale (i.e., NP-Physician Relations, NP-Administration Relations, Professional
Visibility, and Independent Practice and Support) moderates the relationship between NP
burnout and quality of care. Using data from the same sample of primary care NPs as in studies
two and three, we first developed a multi-level proportional odds cumulative logit model and
found that the odds of perceiving higher quality of care was 85% less for NPs with burnout
compared to those without burnout. To test for moderation, we ran multi-level proportional odds
cumulative logit models for each practice environment subscale and found that the practice
environment did not moderate the relationship between burnout and quality of care. However,
with a one unit increase in the mean Professional Visibility, Independent Practice and Support,
NP-Physician Relations, and NP-Administration Relations subscales, the odds of perceiving
112
higher quality of care increased to 4.39, 7.57, 4.76, and 3.83 times, respectively. As a result, NPs
working in favorable practice environments also perceive delivering higher quality of patient
care.
6.11 Strengths and Limitations
This dissertation has various strengths. To begin with, our systematic review is currently
the first and only comprehensive review investigating predictors and outcomes of burnout
inclusive of all PCPs in the U.S. In this review, we first published a study protocol in accordance
with PRISMA guidelines, conducted an extensive literature search twice, and to reduce the
potential for bias- two researchers independently appraised the quality of the included studies,
extracted information from each study, and then compared results (Abraham et al., 2019). Given
our rigorous approach, our review contained studies with a large representative sample of PCPs
(Edwards et al., 2018; Yoon et al., 2017) practicing in a variety of settings including rural and
urban locations as well as in Veteran Affairs primary care clinics (Edwards et al., 2017; Helfrich
et al., 2014).
Additional strengths are noted within studies two to four of this dissertation. For
example, we were able to determine a prevalence of burnout specifically among NPs which is
useful since some researchers (Edwards et al., 2018) have only reported combined burnout
prevalence rates for NPs and physician assistants. Second, we provide valuable evidence on
factors associated with NP burnout in primary care practices. Third, we used data that came from
a large sample of primary care NPs and this dataset is unique because it is the only dataset
containing variables measuring NP burnout, practice environment, use of multifunctional EHRs,
and quality of care in the primary care setting. Fourth, we used rigorous methods involving
multiple imputation analyses to address the problem of missing data and multi-level modeling to
113
account for the clustering effect of NPs within their practices. Lastly, our findings are timely
since administrators and clinicians are calling for greater investigation into the factors associated
with provider burnout to ultimately reduce burnout (Eadie, 2020).
Despite these strengths, this dissertation had several limitations. First, the systematic
review was focused only on PCPs in the U.S., which may not be generalizable to PCPs in other
countries. Second, studies two to four of this dissertation involved a sample of primary care NPs
from only two states, New Jersey and Pennsylvania, in the Northeast which limits the
generalizability of the results to other U.S. states and geographic regions in the West, Midwest
and South. Third, the use of survey data where respondents are asked to self-report on their
behaviors may be subjected to response bias, even though NPs were informed that their
responses would remain confidential. Fourth, important EHR covariates (e.g., NP training and
experience with the EHR, organizational support available to NPs using the EHR, provider
satisfaction with the EHR) were not included in study three because these covariates were not
available from the parent study. As a result, our findings regarding the association between use
of multifunctional EHRs and NP burnout may be oversimplified. Fifth, although nurse
perceptions of quality of care have been shown to align with patient outcomes in the acute care
setting (McHugh & Stimpfel, 2012), and although researchers have successfully measured
perceptions of care quality among primary care physicians (Pantell et al., 2019), it remains
unknown whether NP perceptions of care quality align with patient perceptions of care quality
specifically in the primary care setting.
6.12 Recommendations for Research, Practice, and Policy
This dissertation also has several research, practice, and policy implications.
114
Research Implications: Since we found a significant association between primary care
practice environments and NP burnout, future qualitative studies should be conducted to gather
rich data on what can be modified in the practice environment to reduce NP burnout and improve
the quality of patient care. Additionally, further research with various measures of the EHR are
needed to better determine what features of the EHR contribute to NP burnout. Such studies can
be enhanced through use of focus group interviews which would specifically ask primary care
NPs about their experiences with the EHR. Research using large studies with national samples of
NPs are also needed to see if primary care NP burnout is associated with objective, measurable
patient outcomes such as mortality and hospitalizations. In the future, it should be tested whether
differences exist in the multi-level regression analyses when using a multiply imputed dataset
that was created using a single statistical software compared to two separate software. Lastly,
future research using a retrospective analysis of primary care NP perceptions of care quality in
the context of patient outcomes and perceptions of care are needed to further support the
appropriateness of using measures such as NP perceptions of care quality.
Practice Implications: Since in our systematic review we found that a high workload for
PCPs contributed to PCP burnout, practice administrators may consider reducing a PCP’s
workload as a potential means of reducing burnout. One improvement that can be made within
primary care teams to reduce PCP workload might be through NP-physician co-management,
which involves two PCPs (i.e., NP and physician) sharing the responsibility of patient care tasks
(Norful et al., 2018). Using 26 interviews of primary care NPs and physicians, researchers have
found that successful NP-physician co-management helped to reduce PCP workload which may
help to reduce PCP burnout in the primary care setting (Norful et al., 2018). As a result,
115
clinicians and practice administrators may consider incorporating NP-physician co-management
within their primary care teams.
With nearly 25% of primary care NPs in New Jersey and Pennsylvania experiencing
burnout, clinicians and practice administrators may consider NP burnout a high priority issue
since burnout has been shown to be associated with lower perceptions of care quality for
patients. One way to address provider burnout may be through improvements in the primary care
practice environment. For example, favorable relations with administrators and physicians may
reduce NP burnout. Additionally, practice administrators can promote greater professional
visibility for NPs- one that would allow administrative staff and physicians to have a better
understanding of what NPs are capable of doing in their practice. Practice administrators can also
ensure that their practices have enough resources for NPs to deliver patient care since a lack of
support and resources may contribute to burnout. Moreover, practice administrators should
provide support and resources (e.g., additional training and education with features of the EHR)
to PCPs especially during the implementation of EHRs since in our systematic review, it was
determined that a lack of support with the EHR can lead to PCP burnout.
Policy Implications: Clinicians as well as practice administrators can use the results
from this dissertation to better understand which factors are associated with primary care NP
burnout and craft appropriate organizational policies designed to reduce NP burnout. For
example, results from our systematic review suggest that organizational policies that seek to
reduce provider workload, ensure an adequate supply of staff and resources, and improve the
primary care practice environment might be some steps that can be taken to reduce provider
burnout. Moreover, New Jersey and Pennsylvania are both states with laws reducing the ability
116
of NPs to practice to the fullest extent of their education (AANP, 2019). As a result, it should be
examined whether such state-level policies have any association with NP burnout.
Conclusion
In conclusion, this dissertation investigated predictors and an outcome associated with
burnout among primary care NPs across two states. Overall, favorable practice environments
were associated with lower risk of NP burnout. The use of multifunctional EHRs did not
contribute to NP burnout however more research using a comprehensive set of EHR-specific
covariates is needed to examine this correlation. Lastly, NP burnout was associated with lower
perceptions of quality of care. However, the practice environment did not moderate the
relationship between burnout and quality of care. Instead, NPs working in favorable practice
environments also reported higher perceptions of care quality. Cumulatively, the studies from
this dissertation reveal that some organizational predictors of burnout are modifiable, thereby
suggesting an opportunity for improvements to be made within primary care practices to
ultimately reduce primary care NP burnout.
117
References
AANP. (2019). State practice environment. https://www.aanp.org/advocacy/state/state-practice-
environment.
AANP. (2019). The state of the nurse practitioner profession: results from the national nurse
practitioner sample survey. https://www.aanp.org/news-feed/nurse-practitioner-role-
continues-to-grow-to-meet-primary-care-provider-shortages-and-patient-demands
AANP. (2020). NP fact sheet. https://www.aanp.org/about/all-about-nps/np-fact-sheet
AANP. (2020). What’s a nurse practitioner (NP)?. https://www.aanp.org/about/all-about-
nps/whats-a-nurse-practitioner
Abdulla, L., Al-Qahtani, D. M., & Al-Kuwari, M. G. (2011). Prevalence and determinants of
burnout syndrome among primary healthcare physicians in Qatar. South African Family
Practice, 53(4), 380-383. doi: 10.1080/20786204.2011.10874118
Abraham, C. M., Zheng, K., & Poghosyan, L. (2019). Predictors and outcomes of burnout among
primary care providers in the United States: A systematic review. Medical Care Research
and Review. https://doi.org/10.1177/1077558719888427
Abraham, C.M., Norful, A.A., Stone, P.W., & Poghosyan, L. (2019). Cost-effectiveness of
advanced practice nurses compared to physician-led care for chronic diseases: A
systematic review. Nursing Economic$, 37(6), 293-305.
Abraham, C.M., Zheng, K., Poghosyan, L. (2018). Predictors and outcomes of burnout among
primary care providers practicing in the United States: A systematic review (Protocol
Registration No. CRD42018115746 in Prospero).
http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42018115746
118
Aiken, L. H., Sermeus, W., Van den Heede, K., Sloane, D. M., Busse, R., McKee, M., Bruyneel,
L., Rafferty, A. M., Griffiths, P., Moreno-Casbas, M. T., Tishelman, C., Scott, A.,
Brzostek, T., Kinnunen, J., Schwendimann, R., Heinen, M., Zikos, D., Sjetne, I. S.,
Smith, H. L., & Kutney-Lee, A. (2012). Patient safety, satisfaction, and quality of
hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and
the United States. BMJ (Clinical research ed.), 344, e1717. doi:
https://doi.org/10.1136/bmj.e1717
Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions.
Thousand Oaks, CA: Sage.
Akinwande, M.O., Dikko, H.G., Samson, A. (2015). Variance inflation factor: as a condition for
the inclusion of suppressor variables in regression analysis. Open Journal of Statistics,
5(7), 754.
Anagnostopoulos, F., Liolios, E., Persefonis, G., Slater, J., Kafetsios, K., & Niakas, D. (2012).
Physician burnout and patient satisfaction with consultation in primary health care
settings: evidence of relationships from a one-with-many design. Journal of Clinical
Psychology in Medical Settings, 19(4), 401-410. doi: 10.1007/s10880-011-9278-8
Arndt, B. G., Beasley, J. W., Watkinson, M. D., Temte, J. L., Tuan, W. J., Sinsky, C. A., &
Gilchrist, V. J. (2017). Tethered to the EHR: primary care physician workload assessment
using EHR event log data and time-motion observations. The Annals of Family
Medicine, 15(5), 419-426.
Auerbach, D. I., Straiger, D. O., & Buerhaus, P. I. (2018). Growing ranks of advanced practice
clinicians-implications for the physician workforce. The New England Journal of
Medicine, 378(25), 2358-2360.
119
Babbott, S., Manwell, L. B., Brown, R., Montague, E., Williams, E., Schwartz, M., Hess, E., &
Linzer, M. (2014). Electronic medical records and physician stress in primary care:
results from the MEMO Study. Journal of the American Medical Informatics
Association: JAMIA, 21(e1), e100–e106. https://doi.org/10.1136/amiajnl-2013-001875
Ball, J., Day, T., Murrells, T., Dall’Ora, C., Rafferty, A. M., Griffiths, P., & Maben, J. (2017).
Cross-sectional examination of the association between shift length and hospital nurses
job satisfaction and nurse reported quality measures. BMC Nursing, 16(1), 26.
Barnes, H., Richards, M. R., McHugh, M. D., & Martsolf, G. (2018). Rural and nonrural primary
care physician practices increasingly rely on nurse practitioners. Health Affairs, 37(6),
908-914. doi: 10.1377/hlthaff.2017.1158
Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator variable distinction in social
psychological research: Conceptual, strategic, and statistical considerations. Journal of
Personality and Social Psychology, 51(6), 1173.
Bilal, A., & Ahmed, H. M. (2017). Organizational structure as a determinant of job burnout: an
exploratory study on Pakistani pediatric nurses. Workplace Health & Safety, 65(3), 118-
128.
Blechter, B., Jiang, N., Cleland, C., Berry, C., Ogedegbe, O., & Shelley, D. (2017). Correlates of
burnout in small independent primary care practices in an urban setting. The Journal of
the American Board of Family Medicine, 31(4), 529-536. doi:
10.3122/jabfm.2018.04.170360
Bodner, T. E. (2008). What improves with increased missing data imputations? Structural
Equation Modeling: A Multidisciplinary Journal, 15(4), 651-675. doi:
10.1080/10705510802339072
120
Bragard, I., Dupuis, G., & Fleet, R. (2015). Quality of work life, burnout, and stress in
emergency department physicians: a qualitative review. European Journal of Emergency
Medicine, 22(4), 227-234. doi: 10.1097/MEJ.0000000000000194
Brigham, T., Barden, C., Dopp, A.L., Hengerer, A., Kaplan, J., Malone, C., Martin, M.,
McHugh, M., & Nora, L.M. (2018). A journey to construct an all-encompassing
conceptual model of factors affecting clinician well-being and resilience. National
Academy of Medicine Perspectives, 1-8. doi: 10.31478/201801b
Buerhaus, P. (2018). Nurse Practitioners: A solution to America’s primary care crisis.
In American Enterprise Institute. https://www.aei.org/research-products/report/nurse-
practitioners-a-solution-to-americas-primary-care-crisis/
Buerhaus, P. I., DesRoches, C. M., Dittus, R., & Donelan, K. (2015). Practice characteristics of
primary care nurse practitioners and physicians. Nursing Outlook, 63(2), 144-153. doi:
10.1016/j.outlook.2014.08.008
Carthon, J. M. B., Brom, H., Poghosyan, L., Daus, M., Todd, B., & Aiken, L. (2020). Supportive
clinical practice environments associated with patient-centered care. The Journal for
Nurse Practitioners, 16(4), 294-298. https://doi.org/10.1016/j.nurpra.2020.01.019
Centers for Disease Control and Prevention. (2014). National Center for Health Statistics-
ambulatory health care data. https://www.cdc.gov/nchs/ahcd/index.htm
Chan, K. S., Gaskin, D. J., McCleary, R. R., & Thorpe Jr, R. J. (2019). Availability of health care
provider offices and facilities in minority and integrated communities in the US. Journal
of Health Care for the Poor and Underserved, 30(3), 986-1000.
121
Chang, C. H., Stukel, T. A., Flood, A. B., & Goodman, D. C. (2011). Primary care physician
workforce and medicare beneficiaries' health outcomes. JAMA, 305(20), 2096-2104. doi:
10.1001/jama.2011.665
Cimiotti, J. P., Aiken, L. H., Sloane, D. M., & Wu, E. S. (2012). Nurse staffing, burnout, and
health care–associated infection. American Journal of Infection Control, 40(6), 486-490.
Clark, R. R., & Lake, E. (2020). Burnout, job dissatisfaction and missed care among maternity
nurses. Journal of Nursing Management. doi: 10.1111/jonm.13037
Corboda, L., Tamayo, J.A., Gonzalez, M.A., Martinez, M.L., Rosales, A., & Barbato, S.H.
(2011). Adaptation and validation of the maslach burnout inventory-human services
survey in Cali, Colombia. Colombia Medica, 42, 286-293.
http://colombiamedica.univalle.edu.co/index.php/comedica/article/view/874/1489
Corley, J. (2017). Advanced-practice providers are key to America’s healthcare future.
https://www.forbes.com/sites/realspin/2017/03/16/advanced-practice-providers-are-key-
to-americas-healthcare-future/#36601fe65998
Dall, T., West, T., Chakrabarti, R., Reynolds, R., & Iacobucci, W. (2018). The complexities of
physician supply and demand: projections from 2016 to 2030. https://aamc-
black.global.ssl.fastly.net/production/media/filer_public/bc/a9/bca9725e-3507-4e35-
87e3-d71a68717d06/aamc_2018_workforce_projections_update_april_11_2018.pdf
de Cordova, P. B., Johansen, M. L., Riman, K. A., & Rogowski, J. (2018). Public reporting of
cardiac outcomes for patients with acute myocardial infarction. Journal of
Cardiovascular Nursing, 1-9. doi: 10.1097/JCN.0000000000000524
DesRoches, C. M., Barrett, K. A., Harvey, B. E., Kogan, R., Reschovsky, J. D., Landon, B. E.,
Casalino, L. P., Shortell, S. M., & Rich, E. C. (2015). The results are only as good as the
122
sample: assessing three national physician sampling frames. Journal of General Internal
Medicine, 30 Suppl 3(Suppl 3), S595–S601. doi: https://doi.org/10.1007/s11606-015-
3380-9
Dewa, C. S., Loong, D., Bonato, S., & Trojanowski, L. (2017). The relationship between
physician burnout and quality of healthcare in terms of safety and acceptability: a
systematic review. BMJ Open, 7(6), e015141.
Dewa, C. S., Loong, D., Bonato, S., Thanh, N. X., & Jacobs, P. (2014). How does burnout affect
physician productivity? A systematic literature review. BMC Health Services
Research, 14(1), 325.
Diaz-Quijano, F. A. (2012). A simple method for estimating relative risk using logistic
regression. BMC Medical Research Methodology, 12(1), 14.
Dillman, D.A., Smyth, J.D., & Christian, L.M. (2014). Internet, Mail, and Mixed-Mode Surveys:
The Tailored Design Method. John Wiley & Sons.
Dillon, E. C., Tai-Seale, M., Meehan, A., Martin, V., Nordgren, R., Lee, T., Nauenberg, T., &
Frosch, D. L. (2020). Frontline perspectives on physician burnout and strategies to
improve well-being: Interviews with physicians and health system leaders. Journal of
General Internal Medicine, 35(1), 261–267. doi: https://doi.org/10.1007/s11606-019-
05381-0
Dolan, E. D., Mohr, D., Lempa, M., Joos, S., Fihn, S. D., Nelson, K. M., & Helfrich, C. D.
(2015). Using a single item to measure burnout in primary care staff: a psychometric
evaluation. Journal of General Internal Medicine, 30(5), 582-587. doi: 10.1007/s11606-
014-3112-6
123
Dyrbye, L. N., Shanafelt, T. D., Johnson, P. O., Johnson, L. A., Satele, D., & West, C. P. (2019).
A cross-sectional study exploring the relationship between burnout, absenteeism, and job
performance among American nurses. BMC Nursing, 18(1), 57.
Eadie, W. (2020). Navigating nurse burnout in response to COVID-19. MedCityNews.
https://medcitynews.com/2020/04/navigating-nurse-burnout-in-response-to-covid-19/
Edwards, S. T., Helfrich, C. D., Grembowski, D., Hulen, E., Clinton, W. L., Wood, G. B., Kim,
L., Rose, D. E., & Stewart, G. (2017). Task delegation and burnout trade-offs among
primary care providers and nurses in veterans affairs patient aligned care teams (VA
PACTs). Journal of the American Board of Family Medicine: JABFM, 31(1), 83–93. doi:
https://doi.org/10.3122/jabfm.2018.01.170083
Edwards, S. T., Marino, M., Balasubramanian, B. A., Solberg, L. I., Valenzuela, S., Springer, R.,
Stange, K. C., Miller, W. L., Kottke, T. E., Perry, C. K., Ono, S., & Cohen, D. J. (2018).
Burnout among physicians, advanced practice clinicians and staff in smaller primary care
practices. Journal of General Internal Medicine, 33(12), 2138–2146. doi:
https://doi.org/10.1007/s11606-018-4679-0
Federal Trade Commission. (2014). Policy perspectives: competition and the regulation of
advanced practice nurses. https://www.ftc.gov/system/files/documents/reports/policy-
perspectives-competition-regulation-advanced-practice-
nurses/140307aprnpolicypaper.pdf
Federation of the State Medical Boards. (2018). Physician wellness and burnout.
http://www.fsmb.org/siteassets/advocacy/policies/policy-on-wellness-and-burnout.pdf
Freudenberger, H. J. (1974). Staff burn-out. Journal of Social Issues, 30(1), 159-165. doi:
https://doi.org/10.1111/j.1540-4560.1974.tb00706.x
124
Friedberg, M. W., Coltin, K. L., Safran, D. G., Dresser, M., Zaslavsky, A. M., & Schneider, E.
C. (2009). Associations between structural capabilities of primary care practices and
performance on selected quality measures. Annals of Internal Medicine, 151(7), 456-463.
Friedberg, M. W., Safran, D. G., Coltin, K. L., Dresser, M., & Schneider, E. C. (2009).
Readiness for the patient-centered medical home: structural capabilities of Massachusetts
primary care practices. Journal of General Internal Medicine, 24(2), 162-169.
Gordon, S. H., Baier, R. R., & Gardner, R. L. (2015). Primary care physicians’ use of electronic
health records in Rhode Island: 2009–2014. Rhode Island Medical Journal, 98(10), 29.
Harrison, K. L., Dzeng, E., Ritchie, C. S., Shanafelt, T. D., Kamal, A. H., Bull, J. H., Tilburt, J.
C., & Swetz, K. M. (2017). Addressing palliative care clinician burnout in organizations:
A workforce necessity, an ethical imperative. Journal of Pain and Symptom
Management, 53(6), 1091–1096. doi: https://doi.org/10.1016/j.jpainsymman.2017.01.007
Haupt, E.F. (2016). Predictors for Florida nurse practitioners’ characterization of organizational
climate. Walden University Scholar Works.
https://pdfs.semanticscholar.org/d015/4722289835565fe78cdc0637bc74d53b1f51.pdf
Helfrich, C. D., Dolan, E. D., Simonetti, J., Reid, R. J., Joos, S., Wakefield, B. J., Schectman, G.,
Stark, R., Fihn, S. D., Harvey, H. B., & Nelson, K. (2014). Elements of team-based care
in a patient-centered medical home are associated with lower burnout among VA primary
care employees. Journal of General Internal Medicine, 29 Suppl 2(Suppl 2), S659–S666.
doi: https://doi.org/10.1007/s11606-013-2702-z
Helfrich, C. D., Simonetti, J. A., Clinton, W. L., Wood, G. B., Taylor, L., Schectman, G., Stark,
R., Rubenstein, L. V., Fihn, S. D., & Nelson, K. M. (2017). The association of team-
specific workload and staffing with odds of burnout among VA primary care team
125
members. Journal of General Internal Medicine, 32(7), 760–766. doi:
https://doi.org/10.1007/s11606-017-4011-4
Hoff, T., Carabetta, S., & Collinson, G. E. (2017). Satisfaction, burnout, and turnover among
nurse practitioners and physician assistants: a review of the empirical literature. Medical
Care Research and Review, 76(1), 3-31. doi: 10.1177/1077558717730157
Howe, J. L., Adams, K. T., Hettinger, A. Z., & Ratwani, R. M. (2018). Electronic health record
usability issues and potential contribution to patient harm. JAMA, 319(12), 1276-1278.
Huang, M. Z., Gibson, C. J., & Terry, A. L. (2018). Measuring electronic health record use in
primary care: a scoping review. Applied Clinical Informatics, 9(01), 015-033.
IBM Corp. (2017). IBM SPSS Statistics for Windows, Version 25.0. Armonk, NY: IBM Corp.
Jenkins, J.A. (2018). Advanced practice nurses play an essential role in healthcare.
https://www.aarp.org/health/health-insurance/info-2018/advanced-practice-nurses-
healthcare.html
Jennings, B. M. (2008). Work stress and burnout among nurses: Role of the work environment
and working conditions. Patient safety and quality: An evidence-based handbook for
nurses. Agency for Healthcare Research and Quality (US).
Jesse, M. T., Abouljoud, M., & Eshelman, A. (2015). Determinants of burnout among transplant
surgeons: a national survey in the United States. American Journal of
Transplantation, 15(3), 772-778.
Jha, A.K., Illiff, A.R., Chaoui, A.A., Defossez, S., Bombaugh, M.C., & Miller, Y.R. (2019). A
crisis in health care: A call to action on physician burnout.
http://www.massmed.org/News-and-Publications/MMS-News-Releases/Physician-
Burnout-Report-2018/
126
Josefsson, K.A., Avby, G., Bäck, M.A., & Kjellström, S. (2018). Workers’ experiences of
healthy work environment indicators at well-functioning primary care units in Sweden: A
qualitative study. Scandinavian Journal of Primary Health Care, 36(4), 406-414.
Kanai-Pak, M., Aiken, L. H., Sloane, D. M., & Poghosyan, L. (2008). Poor work environments
and nurse inexperience are associated with burnout, job dissatisfaction and quality
deficits in Japanese hospitals. Journal of Clinical Nursing, 17(24), 3324-3329.
Kane, L. (2020). Medscape national physician burnout and suicide report 2020: The generational
divide. https://www.medscape.com/slideshow/2020-lifestyle-burnout-6012460#2
Kang, H. (2013). The prevention and handling of the missing data. Korean Journal of
Anesthesiology, 64(5), 402.
Kanter, R.M. (1976). The impact of hierarchical structures on the work behavior of women and
men. Social Problems, 23(4), 415-430.
Kim, L. Y., Rose, D. E., Ganz, D. A., Giannitrapani, K. F., Yano, E. M., Rubenstein, L. V., &
Stockdale, S. E. (2020). Elements of the healthy work environment associated with lower
primary care nurse burnout. Nursing Outlook, 68(1), 14-25.
Kim, L. Y., Rose, D. E., Soban, L. M., Stockdale, S. E., Meredith, L. S., Edwards, S. T.,
Helfrich, C. D., & Rubenstein, L. V. (2018). Primary care tasks associated with provider
burnout: findings from a veterans health administration survey. Journal of General
Internal Medicine, 33(1), 50–56. doi: https://doi.org/10.1007/s11606-017-4188-6
Kravet, S. J., Shore, A. D., Miller, R., Green, G. B., Kolodner, K., & Wright, S. M. (2008).
Health care utilization and the proportion of primary care physicians. The American
Journal of Medicine, 121(2), 142-148.
127
Krisberg, K. (2018). Concerns grow about burnout, stress in health care workers: New demands
adding to burden. The Nation’s Health: A Publication of the American Public Health
Association, 48(8), 1-15.
Kuo, Y. F., Chen, N. W., Baillargeon, J., Raji, M. A., & Goodwin, J. S. (2015). Potentially
preventable hospitalizations in Medicare patients with diabetes: a comparison of primary
care provided by nurse practitioners versus physicians. Medical Care, 53(9), 776-783.
Lake, E. T., Hallowell, S. G., Kutney-Lee, A., Hatfield, L. A., Del Guidice, M., Boxer, B. A.,
Ellis, L. N., Verica, L., & Aiken, L. H. (2016). Higher quality of care and patient safety
associated with better NICU work environments. Journal of Nursing Care Quality, 31(1),
24–32. doi: 10.1097/NCQ.0000000000000146
Lake, E. T., Sanders, J., Duan, R., Riman, K. A., Schoenauer, K. M., & Chen, Y. (2019). A
meta-analysis of the associations between the nurse work environment in hospitals and 4
sets of outcomes. Medical Care, 57(5), 353-361.
Larkin, M. (2018). Physician burnout takes a toll on U.S. patients.
https://www.reuters.com/article/us-health-physicians-burnout/physician-burnout-takes-a-
toll-on-u-s-patients-idUSKBN1F621U
Lazris, A., Roth, A., & Brownlee, S. (2018). No more lip service; it’s time we fixed primary care
(part one). Health Affairs Blog.
https://www.healthaffairs.org/do/10.1377/hblog20181115.750150/full/
Lebares, C. C., Guvva, E. V., Ascher, N. L., O'Sullivan, P. S., Harris, H. W., & Epel, E. S.
(2018). Burnout and stress among US surgery residents: psychological distress and
resilience. Journal of the American College of Surgeons, 226(1), 80-90.
128
Lin, M., Battaglioli, N., Melamed, M., Mott, S. E., Chung, A. S., & Robinson, D. W. (2019).
High prevalence of burnout among US emergency medicine residents: results from the
2017 national emergency medicine wellness survey. Annals of Emergency
Medicine, 74(5), 682-690.
Linzer, M., Manwell, L. B., Williams, E. S., Bobula, J. A., Brown, R. L., Varkey, A. B., Man, B.,
McMurray, J. E., Maguire, A., Horner-Ibler, B., Schwartz, M. D., & MEMO (Minimizing
Error, Maximizing Outcome) Investigators. (2009). Working conditions in primary care:
physician reactions and care quality. Annals of Internal Medicine, 151(1), 28–W9. doi:
https://doi.org/10.7326/0003-4819-151-1-200907070-00006
Liu, Y., & Aungsuroch, Y. (2017). Factors influencing nurse-assessed quality nursing care: A
cross-sectional study in hospitals. Journal of Advanced Nursing, 74(4), 935-945. doi:
10.1111/jan.13507
Martin-Misener, R., Harbman, P., Donald, F., Reid, K., Kilpatrick, K., Carter, N., Bryant-
Lukosius, D., Kaasalainen, S., Marshall, D. A., Charbonneau-Smith, R., & DiCenso, A.
(2015). Cost-effectiveness of nurse practitioners in primary and specialised ambulatory
care: systematic review. BMJ Open, 5(6), e007167. doi: 10.1136/bmjopen-2014-007167
Martsolf, G. R., Ashwood, S., Friedberg, M. W., & Rodriguez, H. P. (2018). Linking structural
capabilities and workplace climate in community health centers. INQUIRY: The Journal
of Health Care Organization, Provision, and Financing, 55, 0046958018794542.
Maslach, C., & Jackson, S.E. (1981). The measurement of experienced burnout. Journal of
Occupational Behaviour, 2, 99-113. doi: https://doi.org/10.1002/job.4030020205
Maslach, C., & Leiter, M.P. (2016). Understanding the burnout experience: Recent research and
its implications for psychiatry. World Psychiatry, 15, 103-111.
129
McCleary, L. (2002). Using multiple imputation for analysis of incomplete data in clinical
research. Nursing Research, 51(5), 339-343.
McHugh, M. D., & Stimpfel, A. W. (2012). Nurse reported quality of care: a measure of hospital
quality. Research in Nursing & Health, 35(6), 566-575. doi: 10.1002/nur.21503
McHugh, M.D., Kutney-Lee, A., Cimiotti, J.P., Sloane, D.M., & Aiken, L.H. (2011). Nurses’
widespread job dissatisfaction, burnout, and frustration with health benefits signal
problems for patient care. Health Affairs, 30, 202-210. doi: 10.1377/hlthaff.2010.0100
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for
systematic reviews and meta-analyses: the PRISMA statement. Annals of Internal
Medicine, 151(4), 264-269. doi: 10.1371/journal.pmed.1000097
Monsalve-Reyes, C. S., San Luis-Costas, C., Gómez-Urquiza, J. L., Albendín-García, L.,
Aguayo, R., & Cañadas-De la Fuente, G. A. (2018). Burnout syndrome and its prevalence
in primary care nursing: A systematic review and meta-analysis. BMC Family
Practice, 19(1), 59.
Moola, S., Munn, Z., Tufanaru, C., Aromataris, E., Sears, K., Sfetcu, R., Currie, M., Qureshi, R.,
Mattis, P., Lisy, K., & Mu, P-F. (2017). Systematic reviews of etiology and risk. In:
Aromataris E, Munn Z (Editors). Joanna Briggs Institute Reviewer's Manual.
https://reviewersmanual.joannabriggs.org/
Moss, J. (2019). Burnout is about your workplace, not your people. Harvard Business Review.
https://hbr.org/2019/12/burnout-is-about-your-workplace-not-your-people
Murray, J. S. (2018). Multiple imputation: A review of practical and theoretical
findings. Statistical Science, 33(2), 142-159.
130
Nantsupawat, A., Nantsupawat, R., Kunaviktikul, W., Turale, S., & Poghosyan, L. (2016). Nurse
burnout, nurse-reported quality of care, and patient outcomes in Thai hospitals. Journal of
Nursing Scholarship, 48(1), 83-90.
National Academy of Medicine. (2018). Action collaborative on clinician well-being and
resilience. https://nam.edu/initiatives/clinician-resilience-and-well-being/
National Academy of Sciences. (2018). Factors affecting clinical well-being and resilience.
https://nam.edu/clinicianwellbeing/wp-content/uploads/2019/07/Factors-Affecting-
Clinician-Well-Being-and-Resilience-July-2019.pdf
National Governors Association. (2012). The role of nurse practitioners in meeting increasing
demands for primary care. https://classic.nga.org/cms/home/nga-center-for-best-
practices/center-publications/page-health-publications/col2-content/main-content-list/the-
role-of-nurse-practitioners.html
NCCPA. (2019). 2017 statistical profile of certified physician assistants by state: An annual
report of the national commission on certification of physician assistants.
http://prodcmsstoragesa.blob.core.windows.net/uploads/files/2017StatisticalProfileofCert
ifiedPhysicianAssistantsbyState.pdf
Nezlek, J. B. (2008). An introduction to multilevel modeling for social and personality
psychology. Social and Personality Psychology Compass, 2(2), 842-860.
Norful, A. A., de Jacq, K., Carlino, R., & Poghosyan, L. (2018). Nurse practitioner–physician
comanagement: a theoretical model to alleviate primary care strain. The Annals of Family
Medicine, 16(3), 250-256.
Noseworthy, J., Madara, J., Cosgrove, D., Edgeworth, M., Ellison, E., Krevans, S., & Harrison,
D. (2017). Physician burnout is a public health crisis: a message to our fellow health care
131
CEOs. Health Affairs Blog.
https://www.healthaffairs.org/do/10.1377/hblog20170328.059397/full/
O’Mahony, N. (2011). Nurse burnout and the working environment. Emergency Nurse, 19(5).
Office of the National Coordinator for Health Information Technology. (2019). What are the
advantages of electronic health records?. https://www.healthit.gov/faq/what-are-
advantages-electronic-health-records
Olayiwola, J. N., Willard-Grace, R., Dubé, K., Hessler, D., Shunk, R., Grumbach, K., &
Gottlieb, L. (2018). Higher perceived clinic capacity to address patients' social needs
associated with lower burnout in primary care providers. Journal of Health Care for the
Poor and Underserved, 29(1), 415-429. doi: 10.1353/hpu.2018.0028
Olds, D. M., Aiken, L. H., Cimiotti, J. P., & Lake, E. T. (2017). Association of nurse work
environment and safety climate on patient mortality: A cross-sectional
study. International Journal of Nursing Studies, 74, 155-161.
Olson, K., Sinsky, C., Rinne, S. T., Long, T., Vender, R., Mukherjee, S., Bennick, M., & Linzer,
M. (2018). Cross-sectional survey of workplace stressors associated with physician
burnout measured by the mini-Z and the maslach burnout inventory. Stress and Health:
Journal of the International Society for the Investigation of Stress, 35(2), 157–175. doi:
https://doi.org/10.1002/smi.2849
Panagioti, M., Geraghty, K., & Johnson, J. (2018). How to prevent burnout in cardiologists? A
review of the current evidence, gaps, and future directions. Trends in Cardiovascular
Medicine, 28(1), 1-7.
Panagioti, M., Geraghty, K., Johnson, J., Zhou, A., Panagopoulou, E., Chew-Graham, C., Peters,
D., Hodkinson, A., Riley, R., & Esmail, A. (2018). Association between physician
132
burnout and patient safety, professionalism, and patient satisfaction: A systematic review
and meta-analysis. JAMA Internal Medicine, 178(10), 1317–1330. doi:
10.1001/jamainternmed.2018.3713
Pantell, M. S., De Marchis, E., Bueno, A., & Gottlieb, L. M. (2019). Practice capacity to address
patients’ social needs and physician satisfaction and perceived quality of care. The
Annals of Family Medicine, 17(1), 42-45.
Patel, R. S., Bachu, R., Adikey, A., Malik, M., & Shah, M. (2018). Factors related to physician
burnout and its consequences: A review. Behavioral Sciences, 8(11), 98.
https://doi.org/10.3390/bs8110098
Peckham, C. (2018). Medscape national physician burnout & depression report 2018.
https://www.medscape.com/slideshow/2018-lifestyle-burnout-depression-6009235
Petterson, S., McNellis, R., Klink, K., Meyers, D., Bazemore, A. (2018). The state of primary
care in the United States: A chartbook of facts and statistics. https://www.graham-
center.org/content/dam/rgc/documents/publications-
reports/reports/PrimaryCareChartbook.pdf
Phillips Jr, R. L., & Bazemore, A. W. (2010). Primary care and why it matters for US health
system reform. Health Affairs, 29(5), 806-810. doi: 10.1377/hlthaff.2010.0020
Poghosyan, L., & Aiken, L. H. (2015). Maximizing nurse practitioners' contributions to primary
care through organizational changes. The Journal of Ambulatory Care
Management, 38(2), 109-117.
Poghosyan, L., & Carthon, J. M. B. (2017). The untapped potential of the nurse practitioner
workforce in reducing health disparities. Policy, Politics, & Nursing Practice, 18(2), 84-
94. doi: 10.1177/1527154417721189
133
Poghosyan, L., Clarke, S. P., Finlayson, M., & Aiken, L. H. (2010). Nurse burnout and quality of
care: Cross-national investigation in six countries. Research in Nursing & Health, 33(4),
288-298.
Poghosyan, L., Ghaffari, A., Liu, J., & Martsolf, G.R. (2020). Primary care nurse practitioners’
roles and work environment in six states with variable scope of practice regulations.
Unpublished manuscript.
Poghosyan, L., Liu, J., Shang, J., & D’Aunno, T. (2017). Practice environments and job
satisfaction and turnover intentions of nurse practitioners: Implications for primary care
workforce capacity. Health Care Management Review, 42(2), 162-171.
Poghosyan, L., Nannini, A., Finkelstein, S. R., Mason, E., & Shaffer, J. A. (2013). Development
and psychometric testing of the nurse practitioner primary care organizational climate
questionnaire. Nursing Research, 62(5), 325-334.
Poghosyan, L., Nannini, A., Stone, P. W., & Smaldone, A. (2013). Nurse practitioner
organizational climate in primary care settings: Implications for professional
practice. Journal of Professional Nursing, 29(6), 338-349.
Poghosyan, L., Shang, J., Liu, J., Poghosyan, H., Liu, N., & Berkowitz, B. (2015). Nurse
practitioners as primary care providers: Creating favorable practice environments in New
York State and Massachusetts. Health Care Management Review, 40(1), 46-55.
Pohl, J., Thomas, A., Bigley, M. B., & Kopanos, T. (2018). Primary care workforce data and the
need for nurse practitioner full practice authority. Health Affairs Blog.
https://www.healthaffairs.org/do/10.1377/hblog20181211.872778/full/
134
Purvanova, R. K., & Muros, J. P. (2010). Gender differences in burnout: A meta-
analysis. Journal of Vocational Behavior, 77(2), 168-185. doi:
http://dx.doi.org/10.1016/j.jvb.2010.04.006
Rabatin, J., Williams, E., Baier Manwell, L., Schwartz, M. D., Brown, R. L., & Linzer, M.
(2016). Predictors and outcomes of burnout in primary care physicians. Journal of
Primary Care & Community Health, 7, 41–43. doi:10.1177/2150131915607799
Ratanawongsa, N., Roter, D., Beach, M. C., Laird, S. L., Larson, S. M., Carson, K. A., &
Cooper, L. A. (2008). Physician burnout and patient-physician communication during
primary care encounters. Journal of General Internal Medicine, 23(10), 1581.
Reith, T. P. (2018). Burnout in United States healthcare professionals: A narrative
review. Cureus, 10(12). doi: 10.7759/cureus.3681
Reschovsky, J. D., Ghosh, A., Stewart, K., & Chollet, D. (2012). Paying more for primary care:
can it help bend the Medicare cost curve?. Issue Brief (Commonwealth Fund), 5, 1-14.
Robertson, S. L., Robinson, M. D., & Reid, A. (2017). Electronic health record effects on work-
life balance and burnout within the I3 population collaborative. Journal of Graduate
Medical Education, 9(4), 479-484. doi: 10.4300/JGME-D-16-00123.1
Rohland, B. M., Kruse, G. R., & Rohrer, J. E. (2004). Validation of a single-item measure of
burnout against the maslach burnout inventory among physicians. Stress and Health:
Journal of the International Society for the Investigation of Stress, 20(2), 75-79. doi:
https://doi.org/10.1002/smi.1002
Rotenstein, L. S., Torre, M., Ramos, M. A., Rosales, R. C., Guille, C., Sen, S., & Mata, D. A.
(2018). Prevalence of burnout among physicians: A systematic review. JAMA Journal of
the American Medical Association, 320, 1131-1150.
135
Rowand, L.C. (2017). Primary care nurse practitioners and organizational culture. Walden
University Scholar Works.
https://pdfs.semanticscholar.org/a1a0/93268088f3557cfe011b6a6dca4ac0b3755d.pdf
Samra, R. (2018). Brief history of burnout. BMJ, 363. doi: https://doi.org/10.1136/bmj.k5268
Scanlon, A., Murphy, M., Tori, K., & Poghosyan, L. (2018). A national study of Australian nurse
practitioners’ organizational practice environment. The Journal for Nurse
Practitioners, 14(5), 414-418.
Schaefer, J. A., & Moos, R. H. (1996). Effects of work stressors and work climate on long-term
care staff's job morale and functioning. Research in Nursing & Health, 19(1), 63-73.
Schaufeli, W. B., Leiter, M. P., & Maslach, C. (2009). Burnout: 35 years of research and
practice. Career Development International, 14(3), 204-220. doi:
10.1108/13620430910966406
Schoen, C., Osborn, R., Squires, D., Doty, M., Rasmussen, P., Pierson, R., & Applebaum, S.
(2012). A survey of primary care doctors in ten countries shows progress in use of health
information technology, less in other areas. Health Affairs, 31(12), 2805-2816.
Shanafelt, T. D., Balch, C. M., Bechamps, G., Russell, T., Dyrbye, L., Satele, D., Collicott, P.,
Novotny, P. J., Sloan, J., & Freischlag, J. (2010). Burnout and medical errors among
American surgeons. Annals of Surgery, 251(6), 995–1000. doi:
10.1097/SLA.0b013e3181bfdab3
Shanafelt, T. D., Dyrbye, L. N., Sinsky, C., Hasan, O., Satele, D., Sloan, J., & West, C. P.
(2016). Relationship between clerical burden and characteristics of the electronic
environment with physician burnout and professional satisfaction. Mayo Clinic
Proceedings, 91(7), 836-848.
136
Shanafelt, T. D., Hasan, O., Dyrbye, L. N., Sinsky, C., Satele, D., Sloan, J., & West, C. P.
(2015). Changes in burnout and satisfaction with work-life balance in physicians and the
general US working population between 2011 and 2014. Mayo Clinic Proceedings,
90(12), 1600-1613.
Shenoi, A. N., Kalyanaraman, M., Pillai, A., Raghava, P. S., & Day, S. (2018). Burnout and
psychological distress among pediatric critical care physicians in the United
States. Critical Care Medicine, 46(1), 116-122.
Shi, L. (2012). The impact of primary care: a focused review. Scientifica. doi:
10.6064/2012/432892
Shieh, G. (2011). Clarifying the role of mean centring in multicollinearity of interaction
effects. British Journal of Mathematical and Statistical Psychology, 64(3), 462-477.
Sillero, A., & Zabalegui, A. (2018). Organizational factors and burnout of perioperative
nurses. Clinical Practice and Epidemiology in Mental Health, 14, 132.
Simonetti, J. A., Sylling, P. W., Nelson, K., Taylor, L., Mohr, D. C., Curtis, I., Schectman, G.,
Fihn, S. D., & Helfrich, C. D. (2017). Patient-centered medical home implementation and
burnout among VA primary care employees. The Journal of Ambulatory Care
Management, 40(2), 158–166. doi: 10.1097/JAC.0000000000000160
Sinsky, C., Colligan, L., Li, L., Prgomet, M., Reynolds, S., Goeders, L., Westbrook, J., Tutty,
M., & Blike, G. (2016). Allocation of physician time in ambulatory practice: A time and
motion study in 4 specialties. Annals of Internal Medicine, 165(11), 753–760.
https://doi.org/10.7326/M16-0961
Soler, J. K., Yaman, H., Esteva, M., Dobbs, F., Asenova, R. S., Katic, M., Ozvacic, Z.,
Desgranges, J. P., Moreau, A., Lionis, C., Kotányi, P., Carelli, F., Nowak, P. R., de
137
Aguiar Sá Azeredo, Z., Marklund, E., Churchill, D., Ungan, M., & European General
Practice Research Network Burnout Study Group. (2008). Burnout in European family
doctors: the EGPRN study. Family Practice, 25(4), 245–265.
https://doi.org/10.1093/fampra/cmn038
Spinelli, W. M., Fernstrom, K. M., Galos, D. L., & Britt, H. R. (2016). Extending our
understanding of burnout and its associated factors: providers and staff in primary care
clinics. Evaluation & the Health Professions, 39(3), 282-298. doi:
10.1177/0163278716637900
Stalpers, D., Kieft, R. A., Van Der Linden, D., Kaljouw, M. J., & Schuurmans, M. J. (2016).
Concordance between nurse-reported quality of care and quality of care as publicly
reported by nurse-sensitive indicators. BMC Health Services Research, 16(1), 120.
Stare, J., & Maucort-Boulch, D. (2016). Odds ratio, hazard ratio and relative
risk. Metodoloskizvezki, 13(1), 59.
Starfield, B., Shi, L., & Macinko, J. (2005). Contribution of primary care to health systems and
health. The Milbank Quarterly, 83(3), 457-502.
StataCorp LP. (2015). STATA statistical software: Release 14. College Station, TX.
Stearns, S., & Benight, C. C. (2016). Organizational factors in burnout and secondary traumatic
stress. In Secondary Trauma and Burnout in Military Behavioral Health Providers (pp.
85-113). Palgrave Macmillan, New York.
Stuart, E. A., Azur, M., Frangakis, C., & Leaf, P. (2009). Multiple imputation with large data
sets: a case study of the children's mental health initiative. American Journal of
Epidemiology, 169(9), 1133-1139.
138
Sun, B. Z., Chaitoff, A., Hu, B., Neuendorf, K., Manne, M., & Rothberg, M. B. (2017).
Empathy, burnout, and antibiotic prescribing for acute respiratory infections: a cross-
sectional primary care study in the US. British Journal General Practice, 67(661), e565-
e571.
Szafran, O., Kennett, S. L., Bell, N. R., & Green, L. (2018). Patients’ perceptions of team-based
care in family practice: access, benefits and team roles. Journal of Primary Health
Care, 10(3), 248-257.
Tawfik, D. S., Phibbs, C. S., Sexton, J. B., Kan, P., Sharek, P. J., Nisbet, C. C., Rigdon, J.,
Trockel, M., & Profit, J. (2017). Factors associated with provider burnout in the
NICU. Pediatrics, 139(5), e20164134. doi: https://doi.org/10.1542/peds.2016-4134
Tawfik, D. S., Profit, J., Morgenthaler, T. I., Satele, D. V., Sinsky, C. A., Dyrbye, L. N., Tutty,
M. A., West, C. P., & Shanafelt, T. D. (2018). Physician burnout, well-being, and work
unit safety grades in relationship to reported medical errors. Mayo Clinic
proceedings, 93(11), 1571–1580. doi: https://doi.org/10.1016/j.mayocp.2018.05.014
The Lancet. (2019). Prioritizing primary care in the USA. Lancet (London,
England), 394(10195), 273. doi: https://doi.org/10.1016/S0140-6736(19)31678-2
Tourigny, L., Baba, V. V., Monserrat, S. I., & Lituchy, T. R. (2019). Burnout and absence
among hospital nurses: an empirical study of the role of context in Argentina. European
Journal of International Management, 13(2), 198-223.
U.S. Department of Health and Human Services, Health Resources and Services Administration.
(2016). National and regional projections of supply and demand for primary care
practitioners: 2013-2025. https://bhw.hrsa.gov/sites/default/files/bhw/health-workforce-
analysis/research/projections/primary-care-national-projections2013-2025.pdf
139
U.S. News and World Reports. (2019). Health care access rankings.
https://www.usnews.com/news/best-states/rankings/health-care/healthcare-access
U.S. News and World Reports. (2019). Health care quality rankings.
https://www.usnews.com/news/best-states/rankings/health-care/healthcare-quality
Van Bogaert, P., Kowalski, C., Weeks, S. M., & Clarke, S. P. (2013). The relationship between
nurse practice environment, nurse work characteristics, burnout and job outcome and
quality of nursing care: a cross-sectional survey. International Journal of Nursing
Studies, 50(12), 1667-1677. doi: 10.1016/j.ijnurstu.2013.05.010
van Weel, C. V., & Kidd, M. R. (2018). Why strengthening primary health care is essential to
achieving universal health coverage. Canadian Medical Association Journal, 190(15),
E463-E466. doi: https://doi.org/10.1503/cmaj.170784
Wen, J., Cheng, Y., Hu, X., Yuan, P., Hao, T., & Shi, Y. (2016). Workload, burnout, and
medical mistakes among physicians in China: A cross-sectional study. Bioscience Trends,
10(1), 27-33. doi: 10.5582/bst.2015.01175
West, C. P., Dyrbye, L. N., & Shanafelt, T. D. (2018). Physician burnout: contributors,
consequences and solutions. Journal of Internal Medicine, 283(6), 516-529. doi:
10.1111/joim.12752
West, C. P., Dyrbye, L. N., Erwin, P. J., & Shanafelt, T. D. (2016). Interventions to prevent and
reduce physician burnout: A systematic review and meta-analysis. The
Lancet, 388(10057), 2272-2281. doi: 10.1016/S0140-6736(16)31279-X
Weston, C., Page, R., Jones-Schubart, K., & Akinlotan, M. (2018). Improving cancer screening
for underserved women through an FNP student-led clinic. The Journal for Nurse
Practitioners, 14(5), e101-e104.
140
Whitebird, R. R., Solberg, L. I., Crain, A. L., Rossom, R. C., Beck, A., Neely, C., Dreskin, M., &
Coleman, K. J. (2017). Clinician burnout and satisfaction with resources in caring for
complex patients. General Hospital Psychiatry, 44, 91–95. doi:
https://doi.org/10.1016/j.genhosppsych.2016.03.004
Wiederhold, B. K., Cipresso, P., Pizzioli, D., Wiederhold, M., & Riva, G. (2018). Intervention
for physician burnout: A systematic review. Open Medicine, 13(1), 253-263.
Willard-Grace, R., Hessler, D., Rogers, E., Dube, K., Bodenheimer, T., & Grumbach, K. (2013).
Team structure and culture are associated with lower burnout in primary care. Journal of
the American Board of Family Medicine, 27, 229-238. doi:
10.3122/jabfm.2014.02.130215
Willard-Grace, R., Knox, M., Huang, B., Hammer, H., Kivlahan, C., & Grumbach, K. (2019).
Burnout and health care workforce turnover. The Annals of Family Medicine, 17(1), 36-
41.
Williams, E. S., Manwell, L. B., Konrad, T. R., & Linzer, M. (2007). The relationship of
organizational culture, stress, satisfaction, and burnout with physician-reported error and
suboptimal patient care: results from the MEMO study. Health Care Management
Review, 32(3), 203-212.
World Health Organization. (2019). Primary health care. https://www.who.int/news-room/fact-
sheets/detail/primary-health-care
Xue, Y., Smith, J. A., & Spetz, J. (2019). Primary care nurse practitioners and physicians in low-
income and rural areas, 2010-2016. JAMA, 321(1), 102-105. doi:
10.1001/jama.2018.17944
141
Yen, P. Y., & Bakken, S. (2012). Review of health information technology usability study
methodologies. Journal of the American Medical Informatics Association, 19(3), 413-
422.
Yoon, J. D., Daley, B. M., & Curlin, F. A. (2017). The association between a sense of calling and
physician well-being: a national study of primary care physicians and
psychiatrists. Academic Psychiatry, 41(2), 167-173. doi: 10.1007/s40596-016-0487-1
Zubatsky, M., Pettinelli, D., Salas, J., & Davis, D. (2018). Associations between integrated care
practice and burnout factors of primary care physicians. Family Medicine, 50(10), 770-
774.
142
Appendix A: Search Strategy for Study 1
Database Search Phrase 1 PubMed (("burnout" OR "Burnout, Professional"[Mesh] OR "burnout" [Tiab] OR
"burn* out")) AND ("Nurse Practitioners"[Mesh] OR "advance* practice nurse*" [Tiab] "advance* practice nurse*" OR "nurse practitioner*" [Tiab] OR "nurse practitioner*" OR "primary care provider*" OR "doctor*" [Tiab] OR "clinician*" [Tiab] OR "clinician*" OR "Physicians"[Mesh] OR "physician*" [Tiab] OR "physician*" OR "physician assistant*" OR "physician assistant*" [Tiab]) AND ("factor*" OR "factor*" [Tiab] OR "lead*" OR "lead*" [Tiab] OR "led*" OR "led*" [Tiab] OR "associate*" OR "associate*" [Tiab] OR "predictor*" OR "predictor*" [Tiab] OR "cause*" [Tiab] OR "cause*" OR "outcome*" OR "outcome*" [Tiab] OR "consequence*" OR "consequence*" [Tiab])) AND (“primary care” OR “primary care” [Tiab]))
2 Embase ('burnout'/exp OR burnout:ti,ab OR 'burn* out'/exp OR burn* out:ti,ab) AND ('clinician*'/exp OR clinician*:ti,ab OR 'physician*'/exp OR physician*:ti,ab OR 'doctor*'/exp OR doctor*:ti,ab OR 'nurse practitioner*'/exp OR 'nurse practitioner*' OR 'physician assistant*'/exp OR 'physician assistant*' OR 'primary care provider*'/exp OR 'primary care provider*' OR 'advance* practice nurse*'/exp OR 'advance* practice nurse*' OR primary AND ('care'/exp OR care) OR nurse* AND practitioner*:ti,ab OR advance* AND practice AND nurse*:ti,ab) AND ('factor*' OR 'cause*' OR 'outcome*' OR 'predictor*' OR 'consequence*' OR 'led*' OR 'lead*' OR 'associate*') ('burnout') AND ('physician*' OR 'nurse practitioner*' OR 'advance* practice nurse*' OR 'general practitioner*' OR 'primary care provider*' OR 'physician assistant*') AND ('primary medical care' OR 'primary health care') AND ('predictors*' OR 'outcomes*' OR 'consequences*' OR 'led*' OR 'lead*' OR 'associate*') 'burn*out' AND 'primary health care'
3 PyschINFO ("burnout" or "burn*out") and ("clinician*" or "doctor*" or "physician*" or "primary care provider*" or "nurse practitioner*" or "advance* practice nurse*" or "physician assistant*") and ("primary care") and ("outcome*" or "predictor*" or "factor*" or "consequence*" or "led*" or "lead*" or "associate*" or "cause*") ((primary care provider*.ti,ab.) OR (nurse practitioner*.ti,ab.) OR (advance* practice nurse*.ti,ab.) OR (physician assistant*.ti,ab.) OR (physician*.ti,ab.) OR (primary care physician*.ti,ab.)) AND ((burn*out.ti,ab.) OR (burned out.ti,ab.)) AND ((factor*.ti,ab.) OR (outcome*.ti,ab.) OR (cause*.ti,ab.) OR (consequence*.ti,ab.) OR (predictor*.ti,ab.) OR (led*.ti,ab.) OR (lead*.ti,ab.) OR (associate*.ti,ab.)) AND (primary care.ti,ab.))
143
4 ProQuest^ ("burnout" OR "burn* out") AND ("nurse practitioner*" OR "advance* practice nurse*" OR "physician*" OR "doctor*" OR "clinician*" OR "physician assistant*"OR "primary care provider*") AND ("primary care") AND ("outcome*" OR "factor*" OR "predictor*" OR "consequence*" OR "associate*" OR "lead*" OR "led*" OR "cause*")
5 Joanna Briggs Institute Evidenced-Based Practice Database
("burnout" or "burn* out") and ("clinician*" or "doctor*" or "physician*" or "primary care provider*" or "nurse practitioner*" or "advance* practice nurse*" or "physician assistant*") and ("primary care") and ("outcome*" or "predictor*" or "factor*" or "consequence*" or "associate*" or "led*" or "lead*" or "cause*") ("burnout" or "burn* out") and ("clinician*" or "doctor*" or "physician*" or "primary care provider*" or "nurse practitioner*" or "advance* practice nurse*" or "physician assistant*") and ("primary care")
6 Ovid Medline
('burnout' or 'burn* out') and 'primary care' and ('physician*' or 'nurse practitioner*' or 'advance* practice nurse*' or 'physician assistant*' or 'clinician*' or doctor*' or primary care provider*') and ('outcome*' or 'predictor*' or 'factor*' or 'consequence*' or 'associate*' or 'led*' or 'lead*' or 'cause*').mp. [mp=title, abstract, original title, name of substance word, subject heading word, floating sub-heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier, synonyms]
7 CINAHL- Cumulative Index to Nursing and Allied Health Literature
((TI burnout AND AB burnout) OR (TI burn* out AND AB burn* out)) AND ((TI nurse practitioner* AND AB nurse practitioner*) OR (TI advance* practice nurse* AND AB advance* practice nurse*) OR (TI physician* AND AB physician*) OR (TI doctor* AND AB doctor*) OR (TI physician assistant* AND AB physician assistant*) OR (TI primary care provider* AND AB primary care provider*) OR (TI clinician* AND AB clinician*) OR ((MH "Nurse Practitioner*")) OR (MH "Physician Assistant*") OR (MH "Physician*")) ("burnout" OR "burn* out") AND ("nurse practitioner*" OR "advance* practice nurse*" OR "physician*" OR "doctor*" OR "clinician*" OR "primary care provider*" OR "physician assistant*") AND ("primary care") AND ("outcome*" OR "factor*" OR "predictor*" OR "consequence*" OR "associate*" or "led*" or "lead*" or "cause*")
8 Cochrane Library
("burnout" or "burn* out") and ("clinician*" or "doctor*" or "physician*" or "primary care provider*" or "nurse practitioner*" or "advance* practice nurse*" or "nurse practitioner*" or "physician assistant*") and ("primary care") and ("factor*" or "predictor*" or "consequence*" or "associate*" or "led*" or "lead*" or "cause*" or "outcome*") ("burnout" or "burn* out") and ("clinician*" or "doctor*" or "physician*" or "primary care provider*" or "nurse practitioner*" or "advance* practice nurse*" or "nurse practitioner*" or "physician assistant*") and ("primary care")
Note. ^The authors applied an additional “burnout” filter to the search in ProQuest; To ensure that all appropriate studies were retrieved, the authors used more than one search phrase for certain databases that were yielding a low number of articles when all search terms were
144
included; The Joanna Briggs Institute of Evidence-Based Practice (JBI EBP) was accessed through the JBI EBP database option within Ovid Medline.
