Embed Size (px)
Citation preview
Nancy Spector, PhD, RN, FAAN
Director, Regulatory Innovations
ATI National Nurse Educator Summit
April 1 & 2, 2019, Savannah Georgia
Consensus on Nursing Education Regulatory Quality Indicators: A
Delphi Study
The National Council of State Boards of Nursing (NCSBN) provides education, service and research through collaborative leadership to promote evidence-based regulatory excellence for patient safety and public protection.
Background of the study• Most BONs approve nursing education
programs.
• BONs are asking for legally defensible metrics to measure, other than first-time NCLEX pass rates.
• Results of this study will be analyzed with those from a larger, quantitative study.
Background of the study
Charge: Establish a set of outcomes and associated metrics to recommend processes to assess nursing education programs.
• Review current literature on program approval metrics and their relevance to public safety.
• Recommend factors in addition to first time NCLEX pass rates that can be used to determine criteria for a legally defensible Board of Nursing’s approval/removal process.
LiteratureOutcomes:
1. Licensure pass rates (accreditors; USDE)
2. Practice readiness* (AAMC, 2014; Benner et al., 2010; Berkow et al., 2008; Dickison et al., 2019; Hayden et al., 2014; Kavanagh & Szweda, 2017; Spector et al., 2015)
[*Definition: newly licensed nurses able to deliver consistent, competent and safe care in predictable situations, with guidance for more complex situations.]
Literature
Outcomes (continued):
3. Employment rates (accreditors; Feeg & Mancino, 2016; Matsudaira, 2016)
4. Graduation/retention/persistence rates (Cook & Hartle, 2011; HERI, 2011; Matsudaira, 2016;Randolph, 2013)
Literature RQIs:
1. Quality clinical experiences (Benner et al., 2010; Hungerford, 2019; Kavanagh & Szweda, 2017; NCSBN, 2006; Odom-Maryon et al., 2018).
2. Student/faculty/program characteristics (Alexander, 2019; Benner et al., 2010; Hooper & Ayars, 2017; Libner & Kubela, 2017; Odom-Maryon et al., 2018; Pitt et al., 2012).
3. National nursing accreditation (Cuff & Perez, 2017; Spector et al., 2018; WHO, 2013)
LiteratureWarning signs – opposite of RQIs
1. Lack of curricular rigor (Hooper & Ayars, 2017)
2. High number of complaints (Alexander, 2019)
3. Insufficient qualified faculty (Alexander, 2019; Hooper & Ayars, 2017)
4. Unwillingness to fail unsafe students (Killam et al., 2011; Luhanga et al., 2014)
Study Questions
1. What are the characteristics of nursing programs that graduate safe and competent nurses (RQIs)?
2. What are the warning signs when a program is beginning to fall below standards?
3. What are the outcome measures used to determine if a program is graduating safe and competent students?
Method
Delphi approach:
• Developed during 1950s cold war – RAND Corporation.
• Assumption: Group opinion more valid than.
• Used successfully for policy and education questions.
Method
• Round I – Qualitative responses
• Round II – Rating the factors
• Rounds III & IV – Consensus on those items where respondents disagree
Method
Sample selection:
1. Education consultants from BONs
2. Educators (list from NCLEX)
3. Clinical nurse educators (list from the Association for Nursing Professional Development)
Method
Inclusion Criteria:
1. Education Consultants – selected all
2. Educators – taught students for the last 2 years; PN educator: BSN; RN educator: master’s
3. Clinical nurse educators – work with new graduates
Method
Procedure:
1. IRB – exempted;
2. Piloted questions with n=10 education consultants, educators, clinical nurse educators.
Method
Procedure:
3. Introductory email sent to sample, via Qualtrics (Utah), inviting to participate if they met the criteria;
4. If they were willing to take part and met criteria, clicked into a demographics survey.
5. Defined consensus as 67% agreement, with interquartile range of 1.0 or below.
Round I
1. What are the characteristics/quality indicators of nursing education programs that graduate safe and competent nurses?
2. What are red flags (warning signs) that indicate a program is falling below the standard of graduating safe and competent nurses?
3. What outcome measures could BONs use to determine if nursing programs are graduating safe and competent nurses?
Round I Analysis
Content analysis carried out three ways for validation:
1) By hand;
2) Use of NVivo software;
3) Use of R, with Latent Dirichlet Allocation (LDA).
Round II
Sent out responses (content analyzed) from Round I, asking for RQIs, red flags (warning signs) and outcomes:
• How important are the RQIs (or red flags or outcomes)?
• Rating from 1=unimportant to 4=very important
Round II Analysis
Statistical analysis conducted:
• SPSS (version 22.0) – simple descriptive statistics, looking for percent who agreed that an item was “important” or “very important.”
• Means and SD calculated; Medians and IQR calculated.
• One-way analysis of variance for differences between the groups on ratings.
Results
Sample:
174/293 educators – 59% response rate
71/125 clinical educators – 57% response rate
50/62 education consultants – 81% response rate
Results
Demographics:
Educators – 93% were female; 72% aged 55 or older; 56% had doctorates; 95% taught for more than 5 years.
Education consultants – 96% were female; 72% aged 55 or older; 50% had doctorates; 54% had more than 5 years experience in regulation.
Results
Demographics:
Clinical nurse educators – 96% female; 48% aged 55 or older; 19% had doctorates; 73% had more than 5 years experience.
Results
• Agreement ranges from 78 – 100%
• None of the ratings had a median rating below 3 (important)
• Interquartile ranges from 0-1
Round III not necessary!
RQIsEvidence-based curriculum that emphasizes quality
and safety standards for patient care – 100%
Evidence-based curriculum that emphasizes critical
thinking and clinical reasoning skills – 99.3%
Faculty are able to role model professional
behaviors – 99.3%
Clinical experiences with actual patients that
prepare students for the reality of clinical practice
– 98.7%
Program has a systematic process in place to
address and remediate student practice errors –
98.7%
Faculty teaching clinical courses demonstrate
current clinical competence – 98.7%
RQIsConsistent administrative leadership in the nursing
program – 98.3%
Collaboration between education and practice to
enhance readiness for practice – 97.7%
Ongoing systematic evaluation of the nursing
program – 97.7%
Institutional administrative support of the nursing
program – 97.3%
Consistently has a pattern of NCLEX pass rates that
meet set standards – 96.3%
Administrative support for ongoing faculty
development – 96.3%
RQIsSignificant opportunities for a variety of clinical
experiences with diverse populations – 95.7%
Consistent full-time faculty, as opposed to
reliance on adjunct faculty – 95%
Quality simulation is used to augment clinical
experiences – 93.3%
Comprehensive student support services – 93%
Program has national nursing accreditation – 84%
Admission criteria that emphasize a background
in the sciences – 90%
Red FlagsLack of consistent and prepared clinical faculty –
100%
Limited clinical experiences that do not prepare
the students for practice – 99%
Poor leadership in the nursing program – 98.7%
Trend of NCLEX pass rates is inconsistent or
decreasing – 96.7%
Complaints to the nursing program or board of
nursing from employers, students or faculty –
94.7%
Pattern of faculty attrition – 94%
Red FlagsPattern of nursing program administrator attrition
– 92.7%
Unwillingness of healthcare institutions to host
clinical experiences for the nursing program’s
students – 92%
Pattern of student attrition – 85%
Curriculum is based on “teaching to the NCLEX” –
81.7%
Over-reliance on simulation to replace clinical
experiences with actual patients – 80%
OutcomesNCLEX pass rates of the nursing program –
97.7%
Relationship the nursing program has with its
clinical partners – 97%
Employer satisfaction with the graduates’
readiness for practice – 94.7%
Graduate preparedness to practice for an
interprofessional environment – 93.7%
OutcomesGraduates’ satisfaction with the nursing program
– 85.4%
Graduation rates of students in the nursing
program – 80.4%
Consistency of graduate employment rates with
regional data on nurse employment rates – 79.3%
History of board of nursing discipline with the
graduates of the nursing program – 78%
Discussion Support for RQIs:
1. Quality clinical experiences (Benner et al., 2010; NCSBN, 2006; Kavanagh & Szweda, 2017; Hungerford, 2019; Odom-Maryon et al., 2018;).
2. Administrative leadership and faculty support (Alexander, 2019; Benner et al., 2010; Hooper & Ayars, 2017; Libner & Kubala, 2017).
3. Faculty ratios and qualifications (Libner & Kubala, 2017; Odom-Maryon, 2018)
DiscussionRQIs (continued):
4. Evidence-based curriculum (Benner et al., 2010; Dickison et al., 2019; QSEN, 2019; Ruschet al., 2019).
Warning Signs (besides those that are the opposite of RQIs):
1. Unable to find clinical experiences (Hooper & Ayars, 2017).
2. Complaints to the Board from students, faculty, employers (Alexander, 2019).
3. Teach to the NCLEX (Candela & Bowles, 2007; Kavanagh & Swzeda, 2017; Barrett et al., 2017).
Conclusions and Implications
Strong agreement among faculty, clinical nurse leaders and regulators on:
1. Quality, hands-on clinical experiences – maybe it’s time to focus on faculty development for providing quality direct care experiences?
2. Meaningful collaboration between practice and education is needed – early on in program planning.
Conclusions and Implications3. Administrator consistency and leadership is
essential.
4. A well-prepared, consistent faculty is essential.
These results will be integrated into the larger, 5-Year Annual Report Study for evidence-based
recommendations.
Stay Tuned!
“Perfection is not attainable, but if we chase perfection we can catch excellence.”
- Vince Lombardi
