Academic Program : Assessment Report 2011-2012 Mary Black School of Nursing€¦ · · 2017-09-01Academic Program : Assessment Report 2011-2012 ... a competent provider of nursing

Academic Program: Assessment Report 2011-2012 Unit/Department: Mary Black School of Nursing Division:

P&OD 2.11.10 1

Component Description

Program Mission Statement

From your Program Assessment Plan (Statement should articulate the unit/ program mission in support of the institutional mission and include a clearly defined purpose appropriate to collegiate education.)

MISSION: The primary mission of the Mary Black School of Nursing as part of the University of South Carolina Upstate is to serve the citizens of the Upstate South Carolina by providing education programs in nursing that are of the highest quality. Programs offered at the Mary Black School of Nursing use a variety of teaching modalities and foster inter-institutional articulation for students who are diverse in background, age, race, ethnicity, gender, educational experiences, and needs. Programs are founded upon strong partnerships with the community, including health care organizations and health care providers in Upstate South Carolina.

The faculty of Mary Black School of Nursing is committed to excellence in teaching, advising, and in providing experiential learning opportunities that empower students to become competent professionals who give high quality nursing care to diverse populations. The faculty provides leadership in addressing nursing educational needs and in promoting the health and welfare of the citizens of Upstate South Carolina

Goal 1

From your Program Assessment Plan (Describe broad learning outcomes and concepts (what you want students to learn) expressed in general terms (clear communication, problem-solving skills, etc). Goals should focus on discipline-specific outcomes relevant to the program.)

Use the nursing process to manage the care of clients with acute and chronic health alterations to maximize quality of life and optimize their levels of function and to assist with end-of life care. Reference: MBSON Student Learning Outcomes (Undergraduate Program Objectives) # 3

Objectives SLO’s (student learning outcomes)

From your Program Assessment Plan (Describes the specific skills, values and attitudes students should be able to exhibit that reflect the broader goals. Objectives (student learning outcomes) transform the general program goals into specific student performance/behaviors that demonstrate student learning and skill development along these goals.

1.1 The student will be a competent provider of nursing care who delivers safe, holistic, therapeutic interventions to individuals, families, and communities in a variety of clinical settings

Assessment Methods

From your Program Assessment Plan (Describes the measure(s) by which the department will know the students are meeting the departmental learning objectives. Includes both direct and indirect assessment. Each SLO should have at least one assessment method.)

1.1 EVOLVE© (HESI) RN Exit Exam (4 year track students),

Practicum Notebook from SBSN 461P (RN-BSN track students)


P&OD 2.11.10 2

Assessment Criteria

Level of achievement you are targeting (Indicate benchmarks, scores on assessment instruments, etc… that would indicate acceptable achievement under your plan)

Therapeutic nursing interventions will be assessed through the EVOLVE© (HESI) RN Exit Exam for the 4-Year Track student. The test will be administered the last semester of the senior year in SBSN 497 Issues in Professional Nursing. EVOLVE© (HESI) defines the recommended performance on EVOLVE© (HESI) RN Exit Exam as scoring at, or above, a 900 composite score. The acceptable performance is defined as scoring at, or above, an 850 composite score. The recommended and acceptable student performance scores are based on the HESI Predictability Model (HPM) of student success on (passing) the National Council Licensure Examination for Registered Nurses (NCLEX-RN®) examination. After graduation, students must pass the NCLEX-RN® to enter into the practice of nursing. It is regulated by the National Council of State Boards of Nursing (NCSBN®) and measures the competencies needed to perform safely and effectively as a newly licensed, entry-level registered nurse. In measuring competency in the nursing process and care, the following student outcomes will be measured and analyzed. 1. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National

Mean Composite score on the EVOLVE© (HESI) RN Exit Exam *. 2. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National

Mean Composite score on the therapeutic nursing interventions items on the EVOLVE© (HESI) RN Exit Exam *.

3. Ninety percent of students will obtain a minimum HESI Composite score (HESIScore) of 850 on the EVOLVE© (HESI) RN Exit Exam *.

Students will be provided the opportunity to take the EVOLVE© (HESI) RN Exit Exam three times in order to achieve a minimum HESIScore of 850. Three versions will be available.

Therapeutic nursing interventions have been assessed from the SBSN 461P Community & Public Health Nursing Practicum Notebook grades for the RN to BSN Track students. The notebook is a compilation of clinical practice materials and is graded by course faculty. Rubrics were developed for evaluation of the notebooks to ensure consistent grading among course faculty. Notebook grades will be forwarded to the Associate Dean for tabulation and then be forwarded to the Curriculum Committee for analysis. Satisfactory completion of therapeutic nursing interventions outcome is defined as 85% of RN to BSN Track students will achieve at least a grade of B (85) on the practicum notebook.

The Associate Dean will collect and tabulate the data and forward to the Curriculum Committee for analysis. The Curriculum Committee will analyze the data and present findings along with suggested recommendations to the faculty at a School meeting. Faculty will use assessment findings to evaluate the curriculum and make revisions as indicated. The Assessment Coordinator will collect and correlate the data and prepare the Assessment report in concert with the Strategic Performance Plan Committee.

The faculty will receive the Curriculum Committee’s report at a School meeting and will discuss the implications of the findings as they relate to program improvement. If the outcomes are met at the level established by the faculty, a recommendation to maintain the current standard may be made. If the outcome is not met or partially met, faculty will develop a plan of action to address concerns. The action plan will be implemented and evaluated using the assessment process. Assessment changes will be reflected in the School’s Strategic Performance Plan, documented in the School of Nursing Faculty Meeting minutes, and monitored on an ongoing basis. Ongoing assessment results will be used to evaluate the curriculum and to make program improvements. Findings, analyses, actions, and reevaluations will be documented in the SON Annual Report


P&OD 2.11.10 3

Assessment Results

Actual results and data collected (Make sure to break down data by subgroups (e.g. other campuses or emphases). As appropriate, also include item or category analysis.)

The EVOLVE© (HESI) RN Exit Exam was given to second semester seniors in fall 2011 and spring 2012. The national norm group for this time period consisted of 53,796 nursing students from all types of RN programs (diploma, associate degree, and baccalaureate degree). This number is based on the EVOLVE© (HESI) RN Exit Exam given in the last 365 days. The exam consisted of 150 scored test questions and 10 non-scored test questions. Based on the Kuder-Richardson 20 statistical formula where the KR20 ranges from a -1 to +1, the reliability of the examination (KR20) was 0.84 in December 2010, 0.85 in May 2011, 0.87 in December 2011, and 0.83 in May 2012. The closer the KR20 is to +1, the more internal consistency the exam possesses. A test item question may measure more than one defined category. This is the second year the EVOLVE© (HESI) RN Exit Exam was given at this university. The HESI score also reflects application of the HESI Predictability Model (HPM) to the student composite or aggregate score and is one measure to predict success on the NCLEX-RN® which measures the competencies needed to perform safely and effectively as a newly licensed, entry-level registered nurse. HESI scores range from 0 to over 1,000, and can be as high as 1,500 (depending on the difficulty level of the exam). Research suggests 900 as a recommended score and 850 as an acceptable score.

As a competent provider of care who delivers safe, holistic therapeutic interventions: 1. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National

Mean Composite score on the therapeutic nursing interventions items on the EVOLVE© (HESI) RN Exit Exam *

Category Analysis on EVOLVE© (HESI) RN Exit Exam for Therapeutic Nursing Interventions as Measured by BSN Generalist Nursing Practice Category Student Aggregate HESI Score in Category / Number of Items in Category for 2011 - 2012

Exam Version Dec 2011 Spartanburg Dec 2011 Greenville May 2012 Spartanburg May 2012 Greenville 1 798 / 148 847/ 148 863 / 148 892 / 148 2 882 / 150 893 / 150 811 / 150 822 / 150

The Exam Version was numbered in the order the exams were administered. For example, Version 1 was the first exam administered to students with Version 2 as the second exam administered. The national mean aggregate score among all RNs for 2010-2011 was 840. The assessment measure was met in the May 2011 Spartanburg cohort on Version 2 only. The national mean aggregate score among all RNs for 2011-2012 was 853. The assessment measure was met in the May 2012 for Spartanburg and Greenville cohorts on Version 1 only. The assessment measure was met in December 2011 for the Greenville and Spartanburg cohorts on Version 2 only. This is an improvement from previous years and from the December 2011 cohorts where the measure was met with Version 2 for the Greenville and Spartanburg cohorts.


P&OD 2.11.10 4

Category Analysis on EVOLVE© (HESI) RN Exit Exam for Therapeutic Nursing Interventions as Measured by BSN Generalist Nursing Practice Category Student Aggregate HESI Score in Category / Number of Items in Category for 2010 - 2011

Exam Version Dec 2010 Spartanburg Dec 2010 Greenville May 2011 Spartanburg May 2011 Greenville 1 800 / 148 838 / 148 791 / 147 833 / 147 2 832 / 149 825 / 149 840 / 150 797 / 150 3 738 / 149 700 / 149 795 / 149 751 / 149

Category Analysis on EVOLVE© (HESI) RN Exit Exam for Therapeutic Nursing Interventions Student Aggregate HESI Score for Category / Number of Items in Category for 2009 - 2010

Exam Version Dec 2009 Spartanburg Dec 2009 Greenville May 2010 Spartanburg May 2010 Greenville 1 797 / 135 849 / 135 900 / 131 868 / 131 2 827 / 132 858 / 132 878 / 127 841 / 127 3 775 / 133 809 / 133

1. In comparing the scores from the 2010 – 2011 academic year to the 2011 – 2012 academic year on the therapeutic nursing interventions category, there was an increase in scores in all but the fall 2011 and spring 2012 Version 1 for the Spartanburg cohort. For the 2010 – 2011 academic year there was a decrease in scores in each semester and in each cohort compared to the 2009 – 2010 academic year. It is notable that there were different expectations between the two groups. While the 2009 – 2010 exam held the expectation the student would score at least 900 and was a condition of graduation with the conversion score counting as 10% of the SBSN 497 course grade, the 2010 – 2011 exam policy held the expectation the student would score at least 850 on one of 3 attempts and was not a condition of graduation with the score counting as 30 % of the course grade. The 2009 – 2010 results were based on the NLNAC curriculum category of therapeutic nursing interventions while the 2010 – 2011 and 2011 – 2012 results were based on the AACN curriculum category of BSN Generalist Nursing Practice category. The latter category contains more test items within the category -and could account for this year’s decreases in the 2010 – 2011 academic year. There was an overall increase in scores in the 2011 – 2012 academic year compared to the 2010 to 2011 academic year.

Commented [SA1]: Need to verify based on SBSN 497 policy from syllabus – see p. 8 move from p. 8 here


P&OD 2.11.10 5

2. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the EVOLVE© (HESI) RN Exit Exam *.

The national mean aggregate score among all RNs for 2011-2012 was 853, up from the score of 840 for 2010 -2011 and 823 for 2009-2010. The USC Upstate mean aggregate score (for Version 1) was 799 for the Spartanburg cohort and 846 for the Greenville cohort in fall 2011; 863 in the Spartanburg cohort and 891 in the Greenville cohort for spring 2012. The assessment measure was met for the spring 2012 Greenville and Spartanburg cohorts. The USC Upstate mean aggregate score (for Version 1) in fall 2010 was 838 for the Greenville cohort and was 801 for the Spartanburg cohort and in spring 2011 was 834 for the Greenville cohort and 790 for the Spartanburg cohort. The assessment measure was not met for the 2010 – 2011 year.

In comparison, the USC Upstate mean aggregate score (for Version 1) in fall 2009 was 848 for the Greenville cohort and was 796 for the Spartanburg cohort and in spring 2010 was 865 for the Greenville cohort and 890 for the Spartanburg cohort. The assessment measure was met in all but the fall 2009 semester for the Spartanburg cohort in the 2009 – 2010 academic year.

3. Ninety percent of students will obtain a minimum HESI Composite score (HESIScore) of 850 on the EVOLVE© (HESI) RN Exit Exam *.

Of the 199 students taking the RN Exit Exam for the 2011 – 2012 academic year, 48.7% of students scored 850 or above on Version 1. Of the same 199 students, 72.4% of students obtained a score of 850 or above as the highest score of the exams the student had taken. The assessment measure was not met.

Number & Percent of Students Obtaining (HESI Score) of 850 on the EVOLVE© (HESI) RN Exit Exam * Students reaching

850 Dec 2011

Spartanburg Dec 2011 Greenville

May 2012 Spartanburg

May 2012 Greenville

All Cohorts

On1st Attempt Version 1 18/57 = 31.6% 20/41 = 48.8 % 33/61 = 54.1 % 26/40 = 65 % 97/199 = 48.7%

As Highest Score 38/57 = 66.7% 33/41 = 80.5 % 42/61 = 68.9 % 31/40 = 77.5 % 144/199 = 72.4%

Number & Percent of Students Obtaining (HESIScore) of 850 on the EVOLVE© (HESI) RN Exit Exam * Students reaching

850 Dec 2010



May 2011 Greenville

All Cohorts

On1st Attempt Version 1 17/57 = 29.8% 17/39 = 43.6 % 20/61 = 32.8 % 24/57 = 42.1 % 78/214 = 36.5%

As Highest Score 39/57 = 68.4% 25/39 = 64 % 37/61 = 60.7 % 34/57 = 59.7 % 135/214 = 63.1%

Students could take all 3 comprehensive exit exams. The highest score of the 3 attempts counted as 30 percent of the final 497 grade. If the student’s highest score of the three attempts did not reach the national mean composite score, the student was required to take the National Council of State Boards of Nursing (NCSBN) online course and test. The student’s highest score was calculated in the course grade and was not a required achievement of graduation. EVOLVE© (HESI) computes a "conversion score" defined as a weighted percentage score that considers the average difficulty of the exam and the average difficulty of the test items answered correctly. It is set on a traditional 0-100 scale. The conversion score was used for grade calculation.


P&OD 2.11.10 6

Number of Students Not Obtaining (HESIScore) of 850 As the Highest of the Two Possible Scores on the EVOLVE© (HESI) RN Exit Exam

AND DID Not Pass the NCLEX-RN® Students not scoring 850

Dec 2011 Spartanburg

Dec 2011 Greenville


May 2012 Greenville

All Cohorts

On1st Attempt Version 1

18 of 57 5 fail NCLEX-RN®

5 unknown


1 unknown


2 unknown

26/40 3 fail NCLEX-RN®

3 unknown


11 unknown

As Highest Score


5 unknown


1 unknown


2 unknown


3 unknown


11 unknown

Notes 1 fail NCLEX-RN® w/ scores > 850 on 1st attempt

4 fail NCLEX-RN® w/ scores > 850 on 1st attempt


2 unknown

0 fail NCLEX-RN® w/scores > 850 on 1st attempt

5 fail NCLEX-RN® who had scores > 850 on 1st attempt

w/ 2 unknown

Notes 11 pass NCLEX-RN® w/scores ≤ 850

on 1st attempt

5 pass NCLEX-RN® w/scores ≤ 850

on 1st attempt


on 1st attempt


on 1st attempt


on 1st attempt

Number of Students Not Obtaining (HESIScore) of 850 As the Highest of the Three Possible Scores on the EVOLVE© (HESI) RN Exit Exam



Dec 2010 Greenville


May 2011 Greenville

All Cohorts



5 unknown


1 unknown

20 of 61 2fail NCLEX-RN®

unknown


unknown


unknown

As Highest Score

18 of 57 fail NCLEX-RN®

5 unknown


1 unknown

37of 61 2 fail NCLEX-RN®

12 unknown


5 unknown


23 unknown




2 unknown



w/ 2 unknown

There was an overall decrease in the number of students in 2011 – 2012 who did not obtain the HESIScore of 850 and did not pass the NCLEX-RN® compared to 2010 – 2011. The move to the EVOLVE (HESI) system was driven by the HPM. The results reflect the correlation of the model in predicting NCLEX-RN® pass rates. The EVOLVE (HESI) system also provides a remediation plan based on the student’s individual performance, case studies and practice tests for each specialty exam and the EVOLVE© (HESI) RN Exit Exam. Several of the MBSON specialty courses utilize some portion of these materials, but no course has focused on every aspect of the EVOLVE course content materials.


P&OD 2.11.10 7

SBSN 461P RN-BSN Practicum Notebook (Therapeutic Nursing Interventions) # Completing Notebook # ≥ 85% Percentage Assessment

Summer 2011 70 69 99% MET Summer 2010 64 60 94% MET

Summer 2009 64 62 with 2 incomplete 100% MET

The RN-BSN students again demonstrated a high level of ability to demonstrate therapeutic Nursing interventions in their description of nursing care needed by community health clients.

The above graph shows that there appeared to be a positive relationship beween the highest HESI Score on the EVOLVE© (HESI) RN Exit Exam and NCLEX-RN pass rates from December 2010 to May 2012. The higher the HESIScore (second/red bar), the higher the NCLEX-RN pass rate (third/green bar). There was a marked increase in first attempt HESIScores (first/dark blue bar) and NCLEX-RN pass rates in May 2012 compared to December 2010 – December 2011. The highest increase in first attempt scores on the EVOLVE© (HESI) RN Exit Exam achieved in May 2012 resulted in the highest NCLEX-RN pass rate.

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Dec_2010May_2011

Dec_2011

May_2012

Averages

Graduation Dates

1st Attempt

HighestHESI

NCLEXPass Rate

AVGAdmissingGPA

LowestAdmittingGPA


P&OD 2.11.10 8

The correlation table illustrates that there is a significant strong positive correlation between the highest HESIscore and NCLEX-RN pass rate (r = .934; p = .033), meaning that the higher the HESIscore achieved on a possible three exams, the higher the NCLEX-RN pass rate. There is also a significant positrive correlation between the lowest admitting GPA for nursing students and NCLEX-RN pass rates (r = .911; p = .045). There is a strong positive correlation (r = .806) between the HESIScore achieved on the first attempt and NCLEX-RN pass rates that is not significant (p = .097).

This regression table shows that the highest predictor of NCLEX-RN pass rates was highest HESIscore which accounted for 32.5%; average GPA, which accounted for 11%; and lowest admitting GPA,.which accounted for 6.7% of the NCLEX-RN pass rate. 50.2%.of the NCLEX-RN pass rate can be accounted for by Highest HESIScore, lowest admitting GPA, and average GPA.


P&OD 2.11.10 9

Action Plan What actions or modifications have been or will be made based on this assessment?

Action Plan for 2012 to 2013

1) The HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam specified in the 2010 to 2011 MBSON

Assessment Plan was amended to not have an additional course, NURS 495: Integrated Clinical Concepts (1). Instead, students who were unsuccessful on Version 2 were required to remediate prior to the next scheduled HESI Comprehensive Exam.

2) HESI Exam Policy for EVOLVE© (HESI) RN Comprehensive Exam

a. The HESI Version 1 Comprehensive Exam will be administered after completion of SBSN 450 (Health Alterations III) and SBSN 461Community and Public Health Nursing) and after completion of the Evolve NCLEX-RN® Review Course ..

b. Students must achieve a minimum score of 900 on the HESI exam. If the student does not achieve a minimum score of 900, the student is exempt from HESI Versions 2 and 3.

c. Students may take HESI Comprehensive Exam 2 for additional practice if they score 900 or greater. d. Students must repeat HESI Version 2 exit exam with a minimum score of 850. If the student does

not achieve a minimum score of 850 on Version 2 the student is required to purchase and take HESI Comprehensive Exam 3.

e. The highest score earned on the three attempts will be the recorded grade for the HESI Comprehensive Exam.

If students are unsuccessful on the first or second HESI Comprehensive Exam, they will be required to remediate prior to the next scheduled HESI.

3) Continue to use concept mapping. Conduct faculty development on concept mapping to ensure consistent

use. Explore avenues for measuring the implementation of concept mapping for effectiveness, 4) Consult with Loretta Manning and Lydia Zager to review and analyze examinations in all clinical courses to

ensure they are meeting the identified content of the NCLEX. 5) Integrate simulation intentionally with structured scenarios for meaningful evaluation of student performance. 6) Meet with students during advisement to review individualized HESI remediation plans. Trend student results

on HESI specialty exams to review trajectory towards graduation. Evaluate areas of weakness and assist students in a plan for improvement in those areas.

7) SBSN 308: Pathophysiological Topics in Health Care or SBIO 370: Pathophysiology added Spring 2013 as a requirement for the upper division to help students increase their understanding of disease processes, increase their success in subsequent courses in the program, and increase student performance on HESI tests and the NCLEX – RN (National Council Licensure Exam).

8) Combine didactic and practicum courses to assist students with better integration of the content and application of skills.


P&OD 2.11.10 10

Implementation and Evaluation of Previous Years’ “Action Plan”

How was the action plan identified in the previous year’s report implemented this year, and what was the impact?

The action plan for 2010 – 2012 was evaluated as follows: Implement a mandatory remediation program for students at each level by fall 2011. Remediation will be assigned to course faculty • If a student scores less than 850 on a HESI exam for any specialty course remediation is required. • Students must complete the remediation package provided by Evolve on content missed. • Students will be assigned a faculty mentor who will meet with students either collectively or individually for 4 additional weeks to review content missed and practice questions. This action was implemented as scheduled in fall 2011. The results as noted in the above reveal that this strategy was effective. The number of students scoring below 850, completing remediation, and passing the NCLEX exam on the first attempt increased suggesting the remediation program was an effective strategy. Endorse concept mapping as additional available faculty teaching strategy. This was endorsed in the May faculty organization meeting in Spring 2011 and implemented in fall 2011. The faculty reaffirmed the use of concept mapping as a teaching strategy and for student clinical planning of care in spring 2012. Offer faculty development programs on test writing in fall 2011 • Refer to Appendix B for the Faculty Development Programs • Focus programs on improving teaching strategies and content of the nursing process Provide course coordinators and senior faculty with Remediation Training for Students during Summer 2011 and Fall 2011. This action was completed by fall 2011. Define specific student learning outcomes and measures for nursing care through simulation scenarios. Individual instructors in clinical courses post their simulations, identifying learning outcomes for the scenarios. The students move through the simulation experience, pre-preparatory work, scenario experience, and debriefing. The simulation staff assists the faculty in meeting their goals with their students. The MBSON is a member of Healthcare Simulation of South Carolina (HSSC) which supports and trains faculty in implementing and enhancing learning through simulation (see Appendix B for Faculty Development). The School of Nursing receives the HSSC course catalog on a regular basis that includes all courses and scenarios currently offered and/or projected by HSSC. The scenarios are updated monthly. Create Integrated Clinical Concepts Course (1 hour) for all first semester juniors. • Meet once a week for two hours for the first 8 weeks of semester. • The purpose is to support, motivate, and test students, so that each student who graduates from MBSON would have an excellent likelihood of passing the NCLEX-RN. • Course will combine support for psychological issues (anxiety and negative self-talk) with cognitive preparation (test-taking strategies, practice questions, study skills). Critical thinking has been integrated into SBSN 306 rather than having a separate Integrated Clinical Concepts course. Note: There was improvement in the drug calculation test pass rate fall 2012 as a result of strategy of 5 weekly drug calculation problems solved by students each week.


P&OD 2.11.10 11



Evaluation of the modifications to the HESI policy are as follows: Implement the following policy in fall 2011: HESI Exam Policy for Specialty Exams 1) Students must complete case studies in order to take the HESI specialty exam. 2) Students must achieve a minimum score of 850 on the HESI exam. If the student does not achieve a minimum score of

850, the student will receive a grade of Incomplete for the course until the student: a) successfully completes the HESI remediation and b) remediates with course faculty and c) repeats Version 2 HESI exam

3) The HESI exam grade will be the grade achieved on the last HESI exam taken. 4) If the score is less than 850 on Version 2 HESI and the student achieves a course grade of 81 or less, the student will not

be allowed to progress in the nursing program. This was implemented in fall 2011. The students who scored below 850 usually scored lower on each HESI exam. As a result the student was limited in the time spent on each subject. Item 3 was changed to the highest grade achieved on either of the two exams. This could possibly result in cursory remediation to get the required content covered and calculating the score needed to pass the course may have affected the effort to improve the score on the second exam.HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam 1) The HESI Version 1 exit exam will be administered during week 4 of the semester. 2) Students must achieve a minimum score of 900 on the HESI exam. If the student does not achieve a minimum score of

900, the student must: a) Enroll in NURS 495: Integrated Clinical Concepts (1). b) Repeat HESI Version 2 exit exam with a minimum score of 850. If the student does not achieve a minimum score of

850 the student must: i) Remediate with course faculty and ii) Repeat Version 3 HESI exam at the student’s own expense.

3) If the student earns a score of 850 or greater on Version 2 or 3 the exam grade will be the grade achieved on the last HESI exam taken.

4) If the student earns a score of less than 850 the student will receive a “0” as the exam grade. The HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam was changed in fall 2011 from being given in week 4to being given following the 10-week SBSN 450/450P (Health Alterations III) and SBSN 461/461P (Community and Public Health Nursing) courses as well as the completion of the NCLEX-RN® review course. The HESI scores improved for both of the cohorts in Spring 2012 with an average >850.


P&OD 2.11.10 12


Goal 2


Demonstrate critical thinking in making clinical judgments. Reference: MBSON Learning Outcomes (Undergraduate Program Objectives) # 1



2.1 The student will be a critical thinker who uses clinical judgment and ethical decision making to design/manage/ coordinate nursing care based on evidence and current research, a broad knowledge base and standards of nursing practice.

Assessment Methods


2.1 Critical thinking test items on the EVOLVE© (HESI) RN Exit Exam, Formal Presentation


P&OD 2.11.10 13

Assessment Criteria


Critical thinking will be assessed from the final course grade on a formal presentation in SBSN 461 Community & Public Health Nursing for the RN-BSN Track students by the course faculty. Satisfactory is defined as 85% of RN-BSN students will achieve at least a grade of B (85) on this measure.

Critical thinking will be assessed through the EVOLVE© (HESI) RN Exit Exam for the 4-Year Track student. The test will be administered the last semester of the senior year in SBSN 497 Issues in Professional Nursing. EVOLVE© (HESI) defines the recommended performance on EVOLVE© (HESI) RN Exit Exam as scoring at, or above, a 900 composite score. The acceptable performance is defined as scoring at, or above, an 850 composite score. The recommended and acceptable student performance scores are based on the HESI Predictability Model (HPM) of student success on (passing) the National Council Licensure Examination for Registered Nurses (NCLEX-RN®) examination. After graduation, students must pass the NCLEX-RN® to enter into the practice of nursing. It is regulated by the National Council of State Boards of Nursing (NCSBN®) and measures the competencies needed to perform safely and effectively as a newly licensed, entry-level registered nurse. In measuring competency in the nursing process and care, the following student outcomes will be measured and analyzed. 1. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean

Composite score on the EVOLVE© (HESI) RN Exit Exam *. 2. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean

Composite score on the critical thinking items on the EVOLVE© (HESI) RN Exit Exam *. 3. Ninety percent of students will obtain a minimum HESI Composite score (HESIScore) of 850 on the EVOLVE©

(HESI) RN Exit Exam *.

The curriculum grid outlines course content related to goals and student learning outcomes. All of the goals/outcomes are leveled by academic year and are recurring themes throughout the program of study. Assessment of the student learning outcomes for critical thinking will be conducted as indicated on the curriculum grid.

Learning Outcomes:

1st Semester Junior Year

2nd Semester Junior Year

1st Semester Senior Year

2nd Semester Senior Year

SLO 2.1 Taught (306) Integrated

(All courses)

Integrated (All courses)


Assessed (461) Integrated

(All courses)

The Associate Dean will collect and tabulate the data and forward to the Curriculum Committee for analysis. The Curriculum Committee will analyze the data and present findings along with suggested recommendations to the faculty at a School meeting. Faculty will use assessment findings to evaluate the curriculum and make revisions as indicated. The Assessment Coordinator will collect and correlate the data and prepare the Assessment report in concert with the Strategic Performance Plan Committee.

The faculty will receive the Curriculum Committee’s report at a School meeting and will discuss the implications of the findings as they relate to program improvement. If the outcomes are met at the level established by the faculty, a recommendation to maintain the current standard may be made. If the outcome is not met or partially met, faculty will develop a plan of action to address concerns. The action plan will be implemented and evaluated using the assessment process. Assessment changes will be reflected in the School’s Strategic Performance Plan, documented in the School of Nursing Faculty Meeting minutes, and monitored on an ongoing basis. Ongoing assessment results will be used to evaluate the curriculum and to make program improvements. Findings, analyses, actions, and reevaluations will be documented in the SON Annual Report.


P&OD 2.11.10 14

Assessment Results


The EVOLVE© (HESI) RN Exit Exam was given to second semester seniors in fall 2011 and spring 2012. The national norm group for this time period consisted of 53,796 nursing students from all types of RN programs (diploma, associate degree, and baccalaureate degree). This number is based on the EVOLVE© (HESI) RN Exit Exam given in the last 365 days. The exam consisted of 150 scored test questions and 10 non-scored test questions. Based on the Kuder-Richardson 20 statistical formula where the KR20 ranges from a -1 to +1, the reliability of the examination (KR20) was 0.84 in December 2010, 0.85 in May 2011, 0.87 in December 2011, and 0.83 in May 2012 . The closer the KR20 is to +1, the more internal consistency the exam possesses. A test item question may measure more than one defined category. This is the second year the EVOLVE© (HESI) RN Exit Exam was given at this university. The HESI score also reflects application of the HESI Predictability Model (HPM) to the student composite or aggregate score and is one measure to predict success on the NCLEX-RN® which measures the competencies needed to perform safely and effectively as a newly licensed, entry-level registered nurse. HESI scores range from 0 to over 1,000, and can be as high as 1,500 (depending on the difficulty level of the exam). Research suggests 900 as a recommended score and 850 as an acceptable score.

As a critical thinker who uses clinical judgment and ethical decision making to design/manage/ coordinate nursing care based on evidence and current research, a broad knowledge base and standards of nursing practice:

1. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the critical thinking items on the EVOLVE© (HESI) RN Exit Exam *

Category Analysis on RN Exit (Comprehensive) Exam for Critical Thinking Student Aggregate HESI Score for Category / Number of Items in Category for 2011 - 2012

Exam Version Dec 2011 Spartanburg Dec 2011 Greenville May 2012 Spartanburg May 2012 Greenville 1 797/146 849/146 864/145 893/145 2 885/147 892/147 812/148 818/148

The Exam Version was numbered in the order the exams were administered. For example, Version 1 was the first exam administered to students with Version 2 as the second exam administered. The national mean aggregate score among all RNs for 2010-2011 was 840. The assessment measure was met in the May 2011 Spartanburg cohort on Version 2 only. The national mean aggregate score among all RNs for 2011-2012 was 853. The assessment measure was met in the May 2012 Spartanburg and Greenville cohorts on Version 1 only. This is an improvement from previous years and from the December 2011 cohorts were the measure was met with Version 2.


P&OD 2.11.10 15

Category Analysis on RN Exit (Comprehensive) Exam for Critical Thinking

Student Aggregate HESI Score for Category / Number of Items in Category for 2010 - 2011 Exam Version Dec 2010 Spartanburg Dec 2010 Greenville May 2011 Spartanburg May 2011 Greenville

1 798 / 145 839 / 145 792 / 143 836 / 145 2 834 / 143 823 /143 844 / 145 798 /145 3 738 / 149 762 / 149 795 /149 751 /149

Category Analysis on RN Exit (Comprehensive) Exam for Critical Thinking

Student Aggregate HESI Score for Category / Number of Items in Category for 2009 - 2010

Exam Version Dec 2009 Spartanburg Dec 2009 Greenville May 2010 Spartanburg May 2010 Greenville 1 790 / 145 845 / 145 883 / 143 862 / 143 2 830 / 144 862 / 144 869 / 145 848 / 145 3 777 / 150 806 / 149

1. The category analysis reference for both tables is the AACN curriculum category of critical thinking and most closely matches the student learning outcome of critical thinking. In comparing the scores from the 2009 – 2010 academic year to the 2010 – 2011 academic year on the critical thinking category, there was a decrease in all but for the Spartanburg cohort for Fall 2010 Versions 1 and 2. It is notable there were different expectations between the two groups. While the 2009 – 2010 exam policy held the expectation the student would score at least 900 and was a condition of graduation with the conversion score counting as 10% of the SBSN 497 course grade, the 2010 – 2011 exam policy held the expectation the student would score at least 850 on one of 3 attempts and was not a condition of graduation with the score counting as 30 % of the course grade. There is also a possible influence of fluctuation in class size and entering mean class grade point averages (GPA) for the spring 2011 cohort. The Greenville class cohort increased from 39 in December 2010 to 57 in May 2011. The GPA was lower for the Greenville cohort. In comparing the scores from the 2010 – 2011 academic year to the 2011 to 2012 academic year ,there was an increase in all but the December 2011 version 1 and May 2012 version 2 for the Spartanburg cohort.


P&OD 2.11.10 16

2. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the EVOLVE© (HESI) RN Exit Exam *. The national mean aggregate score among all RNs for 2011-2012 was 853, up from the score of 840 for 2010 -2011 and 823 for 2009-2010. The USC Upstate mean aggregate score (for Version 1) was 799 for the Spartanburg cohort and 846 for the Greenville cohort in fall 2011; 863 in the Spartanburg cohort and 891 in the Greenville cohort for spring 2012. The assessment measure was met for the spring 2012 Greenville and Spartanburg cohorts. The USC Upstate mean aggregate score (for Version 1) in fall 2010 was 838 for the Greenville cohort and was 801 for the Spartanburg cohort and in spring 2011 was 834 for the Greenville cohort and 790 for the Spartanburg cohort. The assessment measure was not met for the 2010 – 2011 year. In comparison, the USC Upstate mean aggregate score (for Version 1) in fall 2009 was 848 for the Greenville cohort and was 796 for the Spartanburg cohort and in spring 2010 was 865 for the Greenville cohort and 890 for the Spartanburg cohort. While the national mean aggregate scores for all RNs increased by 17 points, the USC Upstate MBSON scores increased in 3 of the 4 cohorts. The assessment measure was met in all but the fall 2009 semester for the Spartanburg cohort in the 2009 – 2010 academic year

3. Ninety percent of students will obtain a minimum HESI Composite score (HESIScore) 850 on the EVOLVE© (HESI) RN Exit Exam *.

Of the 214 students taking the RN Exit Exam for the 2010 -2011 academic year, 36.5 % of students scored 850 or above on Version 1. Of the same 214 students, 63.1% of students obtained a score of 850 or above as the highest score of the exams the student had taken. . Of the 199 students taking the RN Exit Exam for the 2011 to 2012 academic year, 48.7% scored 850 or above on Version 1 while 72.4% scored 850 or above as the highest score of the exams the student had taken. This measure was not met; however, students improved from 2010 to 2011 on both versions. Number & Percent of Students Obtaining (HESI Score) of 850 on the EVOLVE© (HESI) RN Exit Exam *

Students reaching 850


Dec 2011 Greenville


May 2012 Greenville

All Cohorts


As Highest Score 38/57 = 66.7% 33/41 = 80.5 % 42/61 = 68.9 % 31/40 = 77.5 % 144/199 = 72.4%

Number & Percent of Students Obtaining (HESIScore) of 850 on the EVOLVE© (HESI) RN Exit Exam *



Dec 2010 Greenville


May 2011 Greenville

All Cohorts


As Highest Score 39/57 = 68.4% 25/39 = 64 % 37/61 = 60.7 % 34/57 = 59.7 % 135/214 = 63.1%

.


P&OD 2.11.10 17

Students could take all 3 comprehensive exit exams. The highest score of the 3 attempts counted as 30 percent of the final 497 grade. If the student’s highest score of the three attempts did not reach the national mean composite score, the student was required to take the National Council of State Boards of Nursing (NCSBN) online course and test. The student’s highest score was calculated in the course grade and was not a required achievement of graduation. EVOLVE© (HESI) computes a "conversion score" defined as a weighted percentage score that considers the average difficulty of the exam and the average difficulty of the test items answered correctly. It is set on a traditional 0-100 scale. The conversion score was used for grade calculation.




Dec 2011 Greenville


May 2012 Greenville

All Cohorts



5 unknown


1 unknown


2 unknown


3 unknown


11 unknown

As Highest Score


5 unknown


1 unknown


2 unknown


3 unknown


11 unknown




2 unknown



w/ 2 unknown

Notes 11 pass NCLEX-RN® w/scores ≤ 850

on 1st attempt


on 1st attempt


on 1st attempt


on 1st attempt


on 1st attempt




Dec 2010 Greenville


May 2011 Greenville

All Cohorts



unknown


unknown


unknown


unknown


unknown

As Highest Score


5 unknown


1 unknown


12 unknown


5 unknown


23 unknown




2 unknown



w/ 2 unknown

There was an overall decrease in the number of students in 2011 – 2012 who did not obtain (HESIScore) of 850 and did not pass the NCLEX-RN® compared to 2010 – 2011.


P&OD 2.11.10 18

The move to the EVOLVE (HESI) system was driven by the HPM. The results reflect the correlation of the model in predicting NCLEX-RN® pass rates. The EVOLVE (HESI) system also provides a remediation plan based on the student’s individual performance, case studies and practice tests for each specialty exam and the EVOLVE© (HESI) RN Exit Exam. Several of the MBSON specialty courses utilize some portion of these materials, but no course has focused on every aspect of the EVOLVE course content materials.

Formal Presentation SBSN 461

Formal Presentation

# Presenting # ≥ 85% Percentage Assessment

Summer 2011 70 70 100% MET Formal Paper SBSN 461

Formal Paper SBSN 461

# Writing # ≥ 85% Percentage Assessment

Summer 2010 64 48 75 % NOT MET Summer 2009 64 60 92.2 % MET

The number of students scoring 85 or above decreased from Summer 2009 to Summer 2010. Students were challenged with the applicability of the Epidemiology paper written in 2009 to their work experience and work areas. In an effort to make the paper more applicable to the student’s clinical practice area, faculty allowed students to choose between two topics. The topics were vulnerable populations or population focused community health care. Rubrics were developed. The students did not follow the rubrics. It is suspected that students were interested in their workplace issues and digressed from writing according to the required criteria that focused on vulnerable populations or population focused community health care to focusing on personal and work issues. Beginning in summer 2011 the RN-BSN students in SBSN 461 gave formal presentations. One hundred percent of the students scored 85 % or above, indicating that this measure was met.


P&OD 2.11.10 19


0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Dec_2010May_2011

Dec_2011

May_2012

Averages

Graduation Dates

1st Attempt

HighestHESI

NCLEXPass Rate

AVGAdmissingGPA

LowestAdmittingGPA


P&OD 2.11.10 20


This regression table shows that the highest predictor of NCLEX-RN pass rates was highest HESIscore which accounted for 32.5%; average GPA, which accounted for 11% and lowest admitting GPA,.which accounted for 6.7% of the NCLEX-RN pass rate. 50.2%.of the NCLEX-RN pass rate can be accounted for by Highest HESIScore, lowest admitting GPA, and average GPA.


P&OD 2.11.10 21



1) The HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam specified in the 2010 to 2011 MBSON Assessment Plan was

amended to not have an additional course, NURS 495: Integrated Clinical Concepts (1). Instead, students who were unsuccessful on Version 2 were required to remediate prior to the next scheduled HESI Comprehensive Exam.


a) The HESI Version 1 Comprehensive Exam will be administered after completion of SBSN 450 (Health Alterations III) and SBSN 461Community and Public Health Nursing) and after completion of the NCLEX-RN® Review Course .

b) Students must achieve a minimum score of 900 on the HESI exam. If the student does not achieve a minimum score of 900, the student is exempt from HESI Versions 2 and 3.

c) Students may take HESI Comprehensive Exam 2 for additional practice if they score 900 or greater. d) Students must repeat HESI Version 2 exit exam with a minimum score of 850. If the student does not achieve a

minimum score of 850 on Version 2 the student is required to purchase and take HESI Comprehensive Exam 3. e) The highest score earned on the three attempts will be the recorded grade for the HESI Comprehensive Exam.

If students are unsuccessful on the first or second HESI Comprehensive Exam, they will be required to remediate prior to the next scheduled HESI.

2) Continue to use concept mapping. Conduct faculty development on concept mapping to ensure consistent use. Explore avenues for measuring the implementation of concept mapping for effectiveness, 3) Consult with Loretta Manning and Lydia Zager to review and analyze examinations in all clinical courses to ensure they

are meeting the identified content of the NCLEX-RN® exam 4) Integrate simulation intentionally with structured scenarios for meaningful evaluation of student performance. 5) Meet with students during advisement to review individualized HESI remediation plans. 6) SBSN 308: Pathophysiological Topics in Health Care or SBIO 370: Pathophysiology added fall 2012 as a requirement for

the upper division to help students increase their understanding of disease processes, increase their success in subsequent courses in the program, and increase student performance on HESI tests and the NCLEX – RN (National Council Licensure Exam).



P&OD 2.11.10 22



The action plan for 2010 – 2012 was evaluated as follows: Implement a mandatory remediation program for students at each level by fall 2011. Remediation will be assigned to course faculty • If a student scores less than 850 on a HESI exam for any specialty course remediation is required. • Students must complete the remediation package provided by Evolve on content missed. • Students will be assigned a faculty mentor who will meet with students either collectively or individually for 4 additional weeks to review content missed and practice questions. This action was implemented as scheduled in fall 2011. The results as noted in the above reveal that this strategy was effective. The number of students scoring below 850, completing remediation, and passing the NCLEX exam on the first attempt increased indicating the remediation program was an effective strategy. Endorse concept mapping as additional available faculty teaching strategy. This was endorsed in the May faculty organization meeting in Spring 2011 and implemented in fall 2011. The faculty reaffirmed the use of concept mapping as a teaching strategy and for student clinical planning of care in spring 2012. Offer faculty development programs on test writing in fall 2011 • Refer to Appendix B for the Faculty Development Programs • Focus programs on improving teaching strategies and content of the nursing process Provide course coordinators and senior faculty with Remediation Training for Students during Summer 2011 and Fall 2011. This action was completed by fall 2011. Define specific student learning outcomes and measures for nursing care through simulation scenarios. Individual instructors in clinical courses post their simulations, identifying learning outcomes for the scenarios. The students move through the simulation experience, pre-preparatory work, scenario experience, and debriefing. The simulation staff assists them in meeting their goals for their students. The MBSON is a member of Healthcare Simulation of South Carolina (HSSC) which supports and trains faculty in implementing and enhancing learning through simulation (see Appendix B for Faculty Development). The School of Nursing receives the HSSC course catalog on a regular basis that includes all courses and scenarios currently offered and/or projected by HSSC. The scenarios are updated monthly. Create Integrated Clinical Concepts Course (1 hour) for all first semester juniors. • Meet once a week for two hours for the first 8 weeks of semester. • The purpose is to support, motivate, and test students, so that each student who graduates from MBSON would have an excellent likelihood of passing the NCLEX-RN. • Course will combine support for psychological issues (anxiety and negative self-talk) with cognitive preparation (test-taking strategies, practice questions, study skills). Critical thinking has been integrated into SBSN 306 rather than having a separate Integrated Clinical Concepts course. Note: There was improvement in the drug calculation test pass rate fall 2012 as a result of strategy of 5 weekly drug calculation problems solved by students each week.


P&OD 2.11.10 23



Evaluation of the modifications to the HESI policy are as follows: Implement the following policy in fall 2011: HESI Exam Policy for Specialty Exams 5) Students must complete case studies in order to take the HESI specialty exam. 6) Students must achieve a minimum score of 850 on the HESI exam. If the student does not achieve a minimum score of



be allowed to progress in the nursing program.

This was implemented in fall 2011. The students who scored below 850 usually scored lower on each HESI exam. As a result the student was limited in the time spent on each subject. Item 3 was changed to the highest grade achieved on either of the two exams. This could possibly result in cursory remediation to get the required content covered and calculating the score needed to pass the course may have affected the effort to improve the score on the second exam. HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam 5) The HESI Version 1 exit exam will be administered during week 4 of the semester. 6) Students must achieve a minimum score of 900 on the HESI exam. If the student does not achieve a minimum score of




8) If the student earns a score of less than 850 the student will receive a “0” as the exam grade.

The HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam was changed in fall 2011 from being given in week 4 to that of following the 10-week SBSN 450/450P (Health Alterations III) and SBSN 461/461P (Community & Public Heatlh Nursing) courses as well as ro the completion of the NCLEX-RN® review course. The HESI scores improved for both of the cohorts in Spring 2012 with an average >850.


P&OD 2.11.10 24


Goal 3


Integrate complex verbal, nonverbal, written communication skills and information technology into professional nursing practice.

Reference: MBSON Learning Outcomes (Undergraduate Program Objectives) # 2



The student will:

3.1 Demonstrate effective communication skills through verbal, nonverbal, written and information technology modes.

3.2 An effective communicator who uses various modalities to provide caring, competent, and holistic nursing care to a diverse population across the lifespan.

Assessment Methods


3.1 Therapeutic Communication on the EVOLVE© (HESI) RN Exit Exam, Online Ethics Presentation

3.2 Therapeutic Communication on the EVOLVE© (HESI) RN Exit Exam, Online Ethics Presentation


P&OD 2.11.10 25

Assessment Criteria


Therapeutic communication will be assessed from the final course grade on an online ethics presentation in SBSN 497: Issues in Professional Nursing Practice for the RN-BSN Track students by the course faculty. Satisfactory is defined as 85% of RN-BSN students will achieve at least a grade of B (85) on this measure. Therapeutic Communication will be assessed through the EVOLVE© (HESI) RN Exit Exam for the 4-Year Track student. The test will be administered the last semester of the senior year in SBSN 497 Issues in Professional Nursing. EVOLVE© (HESI) defines the recommended performance on EVOLVE© (HESI) RN Exit Exam as scoring at, or above, a 900 composite score. The acceptable performance is defined as scoring at, or above, an 850 composite score. The recommended and acceptable student performance scores are based on the HESI Predictability Model (HPM) of student success on (passing) the National Council Licensure Examination for Registered Nurses (NCLEX-RN®) examination. After graduation, students must pass the NCLEX-RN® to enter into the practice of nursing. It is regulated by the National Council of State Boards of Nursing (NCSBN®) and measures the competencies needed to perform safely and effectively as a newly licensed, entry-level registered nurse. In measuring competency in the nursing process and care, the following student outcomes will be measured and analyzed. 1. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above of the National Mean

Composite score on the EVOLVE© (HESI) RN Exit Exam *. 2. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above of the National Mean

Composite score on the therapeutic communication items on the EVOLVE© (HESI) RN Exit Exam *. 3. Ninety percent of students will obtain a minimum HESI Composite score (HESIScore) 850 on the EVOLVE© (HESI)

RN Exit Exam *.

The faculty developed a curriculum grid outlining course content related to goals and student learning outcomes. All of the goals/outcomes are leveled by academic year and are recurring themes throughout the program of study. Assessment of the student learning outcomes for communication will be conducted as indicated on the curriculum grid.

Learning Outcome:





SLO 3.1 Taught (306 Introduction to

Professional Nursing) Integrated (All courses)



Assessed (/497) Integrated (All courses)

SLO 3.2 Taught (306) Integrated (All courses)



Assessed (/497) Integrated (All courses)

The Associate Dean will collect and tabulate the data and forward to the Curriculum Committee for analysis. The Curriculum Committee will analyze the data and present findings along with suggested recommendations to the faculty at a School meeting. Faculty will use assessment findings to evaluate the curriculum and make revisions as identified. The Assessment Coordinator will collect and correlate the data and prepare the Assessment report in concert with the Strategic Performance Plan Committee. The faculty will receive the Curriculum Committee’s report at a School meeting and will discuss the implications of the findings as they relate to program improvement. If the outcomes are met at the level established by the faculty, a recommendation to maintain the current standard may be made. If the outcome is not met or partially met, faculty will develop a plan of action to address concerns. The plan will be implemented and evaluated using the assessment process. Assessment changes will be reflected in the School’s Strategic Performance Plan, documented in the School of Nursing Faculty Meeting minutes, and monitored on an ongoing basis. Ongoing assessment results will be used to evaluate the curriculum and to make program improvements. Findings, analyses, actions, and reevaluations will be documented in the SON Annual Report.


P&OD 2.11.10 26

Assessment Results


The EVOLVE© (HESI) RN Exit Exam was given to second semester seniors in fall 2011 and spring 2012. The national norm group for this time period consisted of 53,796 nursing students from all types of RN programs (diploma, associate degree, and baccalaureate degree). This number is based on the EVOLVE© (HESI) RN Exit Exam given in the last 365 days. The exam consisted of 150 scored test questions and 10 non-scored test questions. Based on the Kuder-Richardson 20 statistical formula where the KR20 ranges from a -1 to +1, the reliability of the examination (KR20) was 0.84 in December 2010, 0.85 in May 2011, 0.87 in December 2011, and 0.83 in May 2012 . The closer the KR20 is to +1, the more internal consistency the exam possesses. A test item question may measure more than one defined category. This is the second year the EVOLVE© (HESI) RN Exit Exam was given at this university. The HESI score also reflects application of the HESI Predictability Model (HPM) to the student composite or aggregate score and is one measure to predict success on the NCLEX-RN® which measures the competencies needed to perform safely and effectively as a newly licensed, entry-level registered nurse. HESI scores range from 0 to over 1,000, and can be as high as 1,500 (depending on the difficulty level of the exam). Research suggests 900 as a recommended score and 850 as an acceptable score.

As an effective communicator who uses various modalities to provide caring, competent, and holistic nursing care to a diverse population across the lifespan.: 1. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean

Composite score on the communication items on the EVOLVE© (HESI) RN Exit Exam *

Category Analysis on RN EVOLVE© (HESI) RN Exit Exam for Therapeutic Communication Student Aggregate HESI Score for Category / Number of Items in Category

Version Category Dec 2011 Spartanburg Dec 2011 Greenville May 2012 Spartanburg May 2012 Greenville 1 Communication 828/22 905/22 862/23 901/23

Interprofessional Communication 811/19 901/19 841/22 892/22

2 Communication 878/27 898/27 835/27 812/27

Interprofessional Communication 879/26 898/26 84424/ 830/24

The Exam Version was numbered in the order the exams were administered. For example, Version 1 was the first exam administered to students with Version 2 as the second exam administered. The national mean aggregate score among all RNs for 2010-2011 was 840. The assessment measure was met in the May 2011 Spartanburg cohort on Version 2 only. The national mean aggregate score among all RNs for 2011-2012 was 853. The assessment measure was met in the May 2012 for Spartanburg cohorts on Version 1 only for all but interprofessional communication. The assessment measure was met for the Greenville cohort in Version 1 for December 2011 and May 2012 for both communication and interprofessional communication.. This is an improvement from previous years and from the December 2011 cohorts were the measure was met with Version 2.


P&OD 2.11.10 27

Category Analysis on RN EVOLVE© (HESI) RN Exit Exam for Therapeutic Communication

Student Aggregate HESI Score for Category / Number of Items in Category

Version Category Dec 2010 Spartanburg Dec 2010 Greenville May 2011 Spartanburg May 2011 Greenville 1 Communication 822 / 22 889 / 22 827 / 24 881 / 24

Interprofessional Communication 822 / 22 889 / 22 821 / 22 874 / 22

2 Communication 818 / 21 861 / 21 840 /20 807 / 20

Interprofessional Communication 818 / 21 861 / 21 840 /20 807 / 20

3 Communication 798 / 23 762 / 23 827 / 25 767 / 25

Interprofessional Communication 798 / 23 762 / 23 827 / 25 767 / 25

The Exam Version was numbered in the order the exams were administered. For example, Version 1 was the first exam administered to students with Version 2 as the second exam administered. The national mean aggregate score among all RNs for 2010-2011 was 840. The assessment measure was met in the May 2011 Spartanburg cohort on Version 2 only.

Category Analysis on RN EVOLVE© (HESI) RN Exit Exam for Therapeutic Communication Student Aggregate HESI Score for Category / Number of Items in Category

ExamVersion Dec 2009 Spartanburg Dec 2009 Greenville May 2010 Spartanburg May 2011 Greenville 1 732 / 12 811 / 12 848 / 14 856 / 14 2 807 / 18 826 / 18 803 / 21 855 / 21 3 772 / 13 801 / 13

1. In comparing the scores from the 2009 – 2010 academic year to the 2010 – 2011 academic year on the therapeutic communication category, there was an overall increase in scores in each semester and in each cohort. It is notable there were different expectations between the two groups. While the 2009 – 2010 exam policy held the expectation the student would score at least 900 and was a condition of graduation with the conversion score counting as 10% of the SBSN 497 course grade, the 2010 – 2011 exam policy held the expectation the student would score at least 850 on one of 3 attempts and was not a condition of graduation with the score counting as 30 % of the course grade. The 2009 – 2010 results were based on the NLNAC curriculum category of therapeutic curriculum while the 2010 – 2011 results were based on the AACN curriculum category of Communication (1998 Essentials) and Professional Communication (2008 Essentials). The latter category contains more test items within the category and could contribute to the mixed results. In comparing the scores from the 2010 to 2011 academic year to the 2011 to 2012 year on the therapeutic communication category, there was an overall increase in scores in the 2011 to 2012 academic year.


P&OD 2.11.10 28

2. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the EVOLVE© (HESI) RN Exit Exam *. The national mean aggregate score among all RNs for 2011-2012 was 853, up from the score of 840 for 2010 -2011 and 823 for 2009-2010. The USC Upstate mean aggregate score (for Version 1) was 799 for the Spartanburg cohort and 846 for the Greenville cohort in fall 2011; 863 in the Spartanburg cohort and 891 in the Greenville cohort for spring 2012. The assessment measure was met for the spring 2012 Greenville and Spartanburg cohorts.

The USC Upstate mean aggregate score (for Version 1) in fall 2010 was 838 for the Greenville cohort and was 801 for the Spartanburg cohort and in spring 2011 was 834 for the Greenville cohort and 790 for theSpartanburg cohort. The assessment measure was not met for the 2010 – 2011 academic year. In comparison, the USC Upstate mean aggregate score (for Version 1) in fall 2009 was 848 for the Greenville cohort and was 796 for the Spartanburg cohort and in spring 2011 was 865 for the Greenville cohort and 890 for the Spartanburg cohort. While the national mean aggregate scores for all RNs increased by 17 points, the USC Upstate MBSON scores increased in 3 of the 4 cohorts.

3. Ninety percent of students will obtain a minimum HESI Composite score (HESIScore) 850 on the EVOLVE© (HESI) RN Exit Exam *. Of the 199 students taking the RN Exit Exam for the 2011 -2012 academic year 48.7 % of students scored 850 or above on Version 1. Of the same 199 students, 72.4 % of students obtained a score of 850 or above as the highest score of the exams the student had taken. This assessment measure was not met.


850 Dec 2011



May 2012 Greenville

All Cohorts


As Highest Score 38/57 = 66.7% 33/41 = 80.5 % 42/61 = 68.9 % 31/40 = 77.5 % 144/199 = 72.4%

Of the 214 students taking the RN Exit Exam for the 2010 -2011 academic year, 36.5 % of students scored 850 or above on Version 1. Of the same 214 students, 63.1% of students obtained a score of 850 or above as the highest score of the exams the student had taken.


850 Dec 2010



May 2011 Greenville

All Cohorts


As Highest Score 39/57 = 68.4% 25/39 = 64 % 37/61 = 60.7 % 34/57 = 59.7 % 135/214 = 63.1%

Students could take all 3 comprehensive exit exams. The highest score of the 3 attempts counted as 30 percent of the final 497 grade. If the student’s highest score of the three attempts did not reach the national mean composite score, the student was required to take the NCSBN online course and test. The student’s highest score was calculated in the course grade and was not a required achievement of graduation. EVOLVE© (HESI) computes a "conversion score" defined as a weighted percentage score that considers the average difficulty of the exam and the average difficulty of the test items answered correctly. It is set on a traditional 0-100 scale. The conversion score was used for grade calculation.


P&OD 2.11.10 29




Dec 2011 Greenville


May 2012 Greenville

All Cohorts



5 unknown


1 unknown


2 unknown


3 unknown


11 unknown

As Highest Score


5 unknown


1 unknown


2 unknown


3 unknown


11 unknown




2 unknown



w/ 2 unknown




Dec 2010 Greenville


May 2011 Greenville

All Cohorts



5 unknown


1 unknown


15 unknown


5 unknown


26 unknown

As Highest Score


5 unknown


1 unknown


12 unknown


5 unknown


23 unknown




2 unknown



w/ 2 unknown

There was an overalll decrease in the number of students in 2011 – 2012 who did not obtain (HESIScore) of 850 and did not pass the NCLEX-RN® compared to 2010 – 2011.


P&OD 2.11.10 30


Formal Online Ethics Presentation SBSN 497 for RN – BSN Track Students # Participating # ≥ 85% Percentage Assessment Summer 2011 75 75 100% Met

Debate/Oral Presentation SBSN 497 for RN – BSN Track Students # Participating # ≥ 85% Percentage Assessment Summer 2010 67 67 100% Met Summer 2009 71 70 98.6 % Met

The RN-BSN students again demonstrated a high level of ability to demonstrate communication. The scores improved the 2010 – 2011 and 2011 – 2012 academic years. One change that may have influenced the increase in percentages was the move of the presentation from a live (in-class) presentation to online Power Point presentations or You Tube video presentations. The RN-BSN students demonstrated a high level of ability to demonstrate communication in summer 2011 since 100% scored 85% or above.


0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Dec_2010May_2011

Dec_2011

May_2012

Averages

Graduation Dates

1st Attempt

HighestHESI

NCLEXPass Rate

AVGAdmissingGPA

LowestAdmittingGPA


P&OD 2.11.10 31


This regression table shows that the highest predictor of NCLEX-RN pass rates was highest HESIscore which accounted for 32.5%; average GPA, which accounted for 11% and lowest admitting GPA,.which accounted for 6.7% of the NCLEX-RN pass rate. 50.2%.of the NCLEX-RN pass rate can be accounted for by Highest HESIScore, lowest admitting GPA, and average GPA.


P&OD 2.11.10 32



1) The HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam specified in the 2010 to 2011 MBSON Assessment Plan

was amended to not have an additional course, NURS 495: Integrated Clinical Concepts (1). Instead, students who were unsuccessful on Version 2 were required to remediate prior to the next scheduled HESI Comprehensive Exam.


a. The HESI Version 1 Comprehensive Exam will be administered after completion of SBSN 450/450P (Health Alterations III) and SBSN 461/461P Community and Public Health Nursing) and after completion of the NCLEX-RN® Review Course.


c. Students may take HESI Comprehensive Exam 2 for additional practice if they score 900 or greater. d. Students must repeat HESI Version 2 exit exam with a minimum score of 850. If the student does not

achieve a minimum score of 850 on Version 2 the student is required to purchase and take HESI Comprehensive Exam 3.


f. If students are unsuccessful on the first or second HESI Comprehensive Exam, they will be required to remediate prior to the next scheduled HESI.

3) Continue to use concept mapping. 4) Consult with Loretta Manning and Lydia Zager to review and analyze examinations in all clinical courses to ensure

they are meeting the identified content of the NCLEX. 5) Integrate simulation intentionally with structured scenarios for meaningful evaluation of student performance. 6) Faculty voted to create the Simulation Standing Committee spring 2012; the committee was previously an ad hoc

Committee. 7) Meet with students during advisement to review individualized HESI remediation plans. 8) SBSN 308: Pathophysiological Topics in Health Care or SBIO 370: Pathophysiology added fall 2012 as a

requirement for the upper division to help students increase their understanding of disease processes, increase their success in subsequent courses in the program, and increase student performance on HESI tests and the NCLEX – RN® Exam.).



P&OD 2.11.10 33



The action plan for 2010 – 2012 was evaluated as follows: Implement a mandatory remediation program for students at each level by fall 2011. Remediation will be assigned to course faculty • If a student scores less than 850 on a HESI exam for any specialty course remediation is required. • Students must complete the remediation package provided by Evolve on content missed. • Students will be assigned a faculty mentor who will meet with students either collectively or individually for 4 additional weeks to review content missed and practice questions. This action was implemented as scheduled in fall 2011. The results as noted in the above reveal that this strategy was effective. The number of students scoring below 850, completing remediation, and passing the NCLEX exam on the first attempt increased indicating the remediation program was an effective strategy. Endorse concept mapping as additional available faculty teaching strategy. This was endorsed in the May faculty organization meeting in Spring 2011 and implemented in fall 2011. The faculty reaffirmed the use of concept mapping as a teaching strategy and for student clinical planning of care in spring 2012. Offer faculty development programs on test writing in fall 2011 • Refer to Appendix B for the Faculty Development Programs • Focus programs on improving teaching strategies and content of the nursing process Provide course coordinators and senior faculty with Remediation Training for Students during Summer 2011 and Fall 2011. This action was completed by fall 2011. Define specific student learning outcomes and measures for nursing care through simulation scenarios. Individual instructors in clinical courses post their simulations, identifying learning outcomes for the scenarios. The students move through the simulation experience, pre-preparatory work, scenario experience, and debriefing. The simulation staff assists them in meeting their goals for their students. The MBSON is a member of Healthcare Simulation of South Carolina (HSSC) which supports and trains faculty in implementing and enhancing learning through simulation (see Appendix B for Faculty Development). The School of Nursing receives the HSSC course catalog on a regular basis that includes all courses and scenarios currently offered and/or projected by HSSC. The scenarios are updated monthly. Create Integrated Clinical Concepts Course (1 hour) for all first semester juniors. • Meet once a week for two hours for the first 8 weeks of semester. • The purpose is to support, motivate, and test students, so that each student who graduates from MBSON would have an excellent likelihood of passing the NCLEX-RN. • Course will combine support for psychological issues (anxiety and negative self-talk) with cognitive preparation (test-taking strategies, practice questions, study skills). Critical thinking has been integrated into SBSN 306 rather than having a separate Integrated Clinical Concepts course. Note: There was improvement in the drug calculation test pass rate fall 2012 as a result of strategy of 5 weekly drug calculation problems solved by students each week.


P&OD 2.11.10 34



Evaluation of the modifications to the HESI policy are as follows: Implement the following policy in fall 2011: HESI Exam Policy for Specialty Exams

9) Students must complete case studies in order to take the HESI specialty exam. 10) Students must achieve a minimum score of 850 on the HESI exam. If the student does not achieve a minimum score of



be allowed to progress in the nursing program.

This was implemented in fall 2011. The students who scored below 850 usually scored lower on each HESI exam. As a result the student was limited in the time spent on each subject. Item 3 was changed to the highest grade achieved on either of the two exams. This could possibly result in cursory remediation to get the required content covered and calculating the score needed to pass the course may have affected the effort to improve the score on the second exam.

HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam 9) The HESI Version 1 exit exam will be administered during week 4 of the semester. 10) Students must achieve a minimum score of 900 on the HESI exam. If the student does not achieve a minimum score of




12) If the student earns a score of less than 850 the student will receive a “0” as the exam grade. The HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam was changed in fall 2011 from being given in week 4 following the 10-week SBSN 450/450P (Health Alterations III) and SBSN 461/461P (Community & Public Heatlh Nursing) courses as well as after the completion This was implemented in fall 2011. The HESI scores improved for both of the cohorts in Spring 2012 with an average >850.


P&OD 2.11.10 35


Goal 4


Demonstrate the behaviors of a professional role model in all modalities of nursing. Reference: MBSON Learning Outcomes (Undergraduate Program Objectives) # 4



4.1. The student will be a professional role model who assumes responsibility and accountability for personal and professional behaviors, ethical practice, and client advocacy.

Assessment Methods


4.1.1. The student aggregate will obtain a minimum HESI Composite score

(HESIScore) of 850 points on the role development portion of the EVOLVE© (HESI) RN Exit Exam *.

4.1.2. 100% of the 4-year and RN-BSN track students will obtain a satisfactory rating

on the final clinical evaluation tool.


P&OD 2.11.10 36

Assessment Criteria


The student as a professional role model will be assessed through the category of role development on the EVOLVE© (HESI) RN Exit Exam for the 4-Year Track students. This test is administered in the final semester of the curriculum. Satisfactory performance on the EVOLVE© (HESI) RN Exit Exam is defined as scoring at, or above, the recommended score of 850.

The faculty developed a curriculum grid outlining course content related to goals and student learning outcomes. All of the goals/outcomes are leveled by academic year and are recurring themes throughout the program of study. Assessment of the student learning outcomes for professional role model will be conducted as indicated on the curriculum grid.

Learning Outcomes:





SLO 2.1 Taught (306) Integrated

(All courses)



Assessed (499P/461P)


For the 4-Year Track student, the Professional Role Model section of the Clinical Evaluation Tool from the SBSN 499P Senior Practicum in Professional Nursing will be utilized. A student must receive a satisfactory rating to be successful in any practicum course. The Senior Practicum will be the course measured since the student must have performed satisfactorily to advance to this level. For the RN-BSN track students, the Nurse Manager Evaluation Tool from the SBSN 435P Professional Nursing Role Transition Practicum course was utilized for the 2009 – 2010 and 2010 – 2011 academic years. For the 2011 – 2012 academic year the assessment measure was a satisfactory rating on the SBSN 461P clinical evaluation tool. The Nurse Manager Evaluation Tool was discontinued as a result of the discontinuation of the clinical component of SBSN 435.

The Associate Dean will collect and tabulate the data and forward to the Curriculum Committee for analysis. The Curriculum Committee will analyze the data and present findings along with suggested recommendations to the faculty at a School meeting. Faculty will use assessment findings to evaluate the curriculum and make revisions as indicated.

The faculty will receive the Curriculum Committee’s report at a School meeting and will discuss the implications of the findings as they relate to program improvement. If the outcomes are met at the level established by the faculty, a recommendation to maintain the current standard may be made. If the outcome is not met or partially met, faculty will develop a plan of action to address concerns. The action plan will be implemented and evaluated using the assessment process. Assessment changes will be reflected in the School’s Strategic Performance Plan, documented in the School of Nursing Faculty Meeting minutes, and monitored on an ongoing basis. Ongoing assessment results will be used to evaluate the curriculum and to make program improvements. Findings, analyses, actions, and reevaluations will be documented in the SON Annual Report.


P&OD 2.11.10 37

Assessment Results


The two outcome measures documents used were:

1. EVOLVE© (HESI) RN Exit Exam 2. Clinical Evaluation Tool

The EVOLVE© (HESI) RN Exit Exam was given to second semester seniors in fall 2011 and spring 2012. The national norm group for this time period consisted of 53,796 nursing students from all types of RN programs (diploma, associate degree, and baccalaureate degree). This number is based on the EVOLVE© (HESI) RN Exit Exam given in the last 365 days. The exam consisted of 150 scored test questions and 10 non-scored test questions. Based on the Kuder-Richardson 20 statistical formula where the KR20 ranges from a -1 to +1, the reliability of the examination (KR20) was 0.84 in December 2010, 0.85 in May 2011, 0.87 in December 2011, and 0.83 in May 2012 . The closer the KR20 is to +1, the more internal consistency the exam possesses. A test item question may measure more than one defined category. This is the second year the EVOLVE© (HESI) RN Exit Exam was given at this university. The HESI score also reflects application of the HESI Predictability Model (HPM) to the student composite or aggregate score and is one measure to predict success on the NCLEX-RN® which measures the competencies needed to perform safely and effectively as a newly licensed, entry-level registered nurse. HESI scores range from 0 to over 1,000, and can be as high as 1,500 (depending on the difficulty level of the exam). Research suggests 900 as a recommended score and 850 as an acceptable score. As a professional role model who assumes responsibility and accountability for personal and professional behaviors, ethical practice, and client advocacy; 1. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the

National Mean Composite score on the role development items on the EVOLVE© (HESI) RN Exit Exam *


P&OD 2.11.10 38

Category Analysis on EVOLVE© (HESI) RN Exit Exam for Role Development Student Aggregate HESI Score for Category / Number of Items in Category’

Version Category Dec 2011 Spartanburg

Dec 2011 Greenville


May 2012 Greenville

1 Professionalism/ Professional Values 702/8 771/8 930/10 899/10

Manager of Care 749/29 842/29 800/27 843/27

Member of Profession 746/5 863/5 Not measured Not measured

Provider of Care 801/136 842/136 867/139 888/139

2 Professionalism/ Professional Values 802/6 822/6 921/5 743/5

Manager of Care 892/33 918/33 843/28 844/28

Member of Profession 381/1 510/1 809/1 726/1

Provider of Care 878/132 888/132 808/136 814/136

The Exam Version was numbered in the order the exams were administered. For example, Version 1 was the first exam administered to students with Version 2 as the second exam administered. The national mean aggregate score among all RNs for 2010-2011 was 840. The assessment measure was met in the May 2011 Spartanburg cohort on Version 2 only. The national mean aggregate score among all RNs for 2011-2012 was 853. The assessment measure was met in the May 2012 for the Spartanburg and Greenville cohorts on Version 1 only for professionalism and provider of care. This is an improvement from previous years and from the December 2011 cohorts where the measure was met with Version 2. The goal was not met for member of the profession which only included 1 test item for Version 2 for the December 2011 and May 2012 exams.


P&OD 2.11.10 39


Version Category Dec 2010 Spartanburg

Dec 2010 Greenville


May 2011 Greenville

1 Professionalism/ Professional Values 698 / 7 823 / 7 788 / 9 842 / 9

Manager of Care 814/29 822 / 29 789 / 28 812 / 28

Member of Profession 617 / 4 719 / 4 710 / 4 743 / 4

Provider of Care 803/135 841 / 135 787 / 138 835 / 138


Manager of Care 875 / 29 835 / 29 881 / 32 837 / 32

Member of Profession 795 /4 831 / 4 750 / 6 762 / 6

Provider of Care 833 / 127 829 / 127 831 / 128 788 / 128


Manager of Care 695 / 30 714 / 30 834 / 12 736 / 12

Member of Profession 903 / 1 826 / 1 852 / 32 767 / 32

Provider of Care 740 / 142 698 / 142 789 / 143 748 / 143

The Exam Version was numbered in the order the exams were administered. For example, Version 1 was the first exam administered to students with Version 2 as the second exam administered. The national mean aggregate score among all RNs for 2010-2011 was 840. The assessment measure was met in the May 2011 Spartanburg cohort on Version 2 only. The national mean aggregate score among all RNs for 2011-2012 was 853. The assessment measure was met in the May 2012 for the Spartanburg and Greenville cohorts on Version 1 only for professionalism and provider of care. This is an improvement from previous years and from the December 2011 cohorts where the measure was met with Version 2. The goal was not met for member of the profession which only included 1 test item for Version 2 for the December 2011 and May 2012 exams.


P&OD 2.11.10 40


Exam Dec 2009 Spartanburg Dec 2009 Greenville May 2010 Spartanburg May 2010 Greenville 1 763–802/173 810–897/173 740–902/174 812–903/174

2 690–821/164 668–861/164 734–867/169 811–888/169

3 768 – 775 / 165 792 – 940 / 170

In comparing the scores from the 2009 – 2010 academic year to the 2010 – 2011 academic year in the role development category, there was an overall decrease in scores in each semester and in each cohort. It is notable there were different expectations between the two groups. However, the results are inconsistent among the cohorts. While the 2009 – 2010 exam policy held the expectation the student would score at least 900 and was a condition of graduation with the conversion score counting as 10% of the SBSN 497 course grade, the 2010 – 2011 exam policy held the expectation the student would score at least 850 on one of 3 attempts and was not a condition of graduation with the score counting as 30 % of the course grade. There is also a possible influence of fluctuation in class size and entering mean class grade point averages (GPA) for the spring 2011 cohort. The Greenville class cohort increased from 39 in December 2010 to 57 in May 2011. The GPA was lower for the Greenville cohort. The 2009 – 2010 results were based on the AACN curriculum categories of manager of care, provider of care, and member of profession while the 2010 – 2011 results were based on the additional AACN curriculum category professionalism/ professional values. The Leadership course moved from an in-class course to on-line during the 2010 – 2011 academic year. In comparing the scores from the 2010 – 2011 to the 2011 – 2012 academic years in the role development category, there was an overall increase in scores in each semester for each cohort in all role development categories in Version 1.

2. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the EVOLVE© (HESI) RN Exit Exam *.

The national mean aggregate score among all RNs for 2011-2012 was 853, up from the score of 840 for 2010 -2011 and 823 for 2009-2010. The USC Upstate mean aggregate score (for Version 1) was 799 for the Spartanburg cohort and 846 for the Greenville cohort in fall 2011; 863 in the Spartanburg cohort and 891 in the Greenville cohort for spring 2012. The assessment measure was met for the spring 2012 Greenville and Spartanburg cohorts. The USC Upstate mean aggregate score (for Version 1) in fall 2010 was 838 for the Greenville cohort and was 801 for the Spartanburg cohort and in spring 2011 was 834 for the Greenville cohort and 790 for the Spartanburg cohort. The assessment measure was not met for the 2010 – 2011 academic year. In comparison, the USC Upstate mean aggregate score (for Version 1) in fall 2009 was 848 for the Greenville cohort and was 796 for the Spartanburg cohort and in spring 2010 was 865 for the Greenville cohort and 890 for the Spartanburg cohort. While the national mean aggregate scores for all RNs increased by 17 points, the USC Upstate MBSON scores increased in 3 of the 4 cohorts.


P&OD 2.11.10 41


850 Dec 2011



May 2012 Greenville

All Cohorts


As Highest Score 38/57 = 66.7% 33/41 = 80.5 % 42/61 = 68.9 % 31/40 = 77.5 % 144/199 = 72.4%

Of the 199 students taking the RN Exit Exam for the 2011 to 2012 academic year, 48.7% scored 850 or above on Version 1 while 72.4% scored 850 or above as the highest score of the exams the student had taken. This measure was not met; however, students improved from 2010 to 2011 on both versions. Of the 214 students taking the RN Exit Exam for the 2010 -2011 academic year, 36.5 % of students scored 850 or above on Version 1. Of the same 214 students, 63.1% of students obtained a score of 850 or above as the highest score of the exams the student had taken. Number & Percent of Students Obtaining (HESIScore) of 850 on the EVOLVE© (HESI) RN Exit Exam *



Dec 2010 Greenville


May 2010 Greenville

All Cohorts


As Highest Score 39/57 = 68.4% 25/39 = 64 % 37/61 = 60.7 % 34/57 = 59.7 % 135/214 = 63.1%

Students could take all 3 comprehensive exit exams. The highest score of the 3 attempts counted as 30 percent of the final 497 grade. If the student’s highest score of the three attempts did not reach the national mean composite score, the student was required to take the NCSBN online course and test. The student’s highest score was calculated in the course grade and was not a required achievement of graduation. EVOLVE© (HESI) computes a "conversion score" defined as a weighted percentage score that considers the average difficulty of the exam and the average difficulty of the test items answered correctly. It is set on a traditional 0-100 scale. The conversion score was the score used in the for grade calculation.




Dec 2011 Greenville


May 2012 Greenville

All Cohorts



5 unknown


1 unknown


2 unknown


3 unknown


11 unknown

As Highest Score


5 unknown


1 unknown


2 unknown


3 unknown


11 unknown




2 unknown



w/ 2 unknown


P&OD 2.11.10 42

Number of Students Not Obtaining (HESIScore) of 850

As the Highest of the Three Possible Scores on the EVOLVE© (HESI) RN Exit Exam AND DID Not Pass the NCLEX-RN®

Students not scoring 850


Dec 2010 Greenville


May 2011Greenville

All Cohorts



5 unknown


1 unknown


15 unknown


5 unknown


26 unknown

As Highest Score


5 unknown


1 unknown


12 unknown


5 unknown


23 unknown




2 unknown



w/ 2 unknown There was an overall decrease in the number of students in 2011 – 2012 who did not obtain (HESIScore) of 850

and did not pass the NCLEX-RN® compared to 2010 – 2011.


3. 100% of the students will obtain a satisfactory rating on the final clinical evaluation tool. Percent of 4-Year Track Students Obtaining a Satisfactory Rating on the Final Clinical Evaluation Tool

Dec 2011 Spartanburg Dec 2011 Greenville May 2012 Spartanburg May 2012 Greenville 4 year Track

Students 100 % 100 % 100 % 100 %

Percent of RN-BSN Track Students Obtaining a Satisfactory Rating on the Final Clinical Evaluation Tool.

Summer 2011 Greenville RN – BSN

Track Students

100% (1 incomplete)

Percent of Students Obtaining a Satisfactory Rating on the Final Clinical Evaluation Tool

Dec 2010 Spartanburg Dec 2010 Greenville May 2011 Spartanburg May 2011 Greenville 4 year Track

Students 100 % 100 % 100 % 100 %

RN – BSN Track Students 100 % 100 % 100 % 100 %

Both the 4-year track and RN-BSN students again demonstrated professional role behaviors in the clinical area in the academic year from 2011 to 2012.


P&OD 2.11.10 43

The above graph shows that there appeared to be a positive relationship beween the highest HESI Score on the

EVOLVE© (HESI) RN Exit Exam and NCLEX-RN pass rates from December 2010 to May 2012. The higher the HESIScore (second/red bar), the higher the NCLEX-RN pass rate (third/green bar). There was a marked increase in first attempt HESIScores (first/dark blue bar) and NCLEX-RN pass rates in May 2012 compared to December 2010 – December 2011. The highest increase in first attempt scores on the EVOLVE© (HESI) RN Exit Exam achieved in May 2012 resulted in the highest NCLEX-RN pass rate.

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Dec_2010May_2011

Dec_2011

May_2012

Averages

Graduation Dates

1st Attempt

HighestHESI

NCLEXPass Rate

AVGAdmissingGPA

LowestAdmittingGPA


P&OD 2.11.10 44


This regression table shows that the highest predictor of NCLEX-RN pass rates was highest HESIscore

which accounted for 32.5%; average GPA, which accounted for 11% and lowest admitting GPA,.which accounted for 6.7% of the NCLEX-RN pass rate. 50.2%.of the NCLEX-RN pass rate can be accounted for by Highest HESIScore, lowest admitting GPA, and average GPA.


P&OD 2.11.10 45



1. The HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam specified in the 2010 to 2011 MBSON

Assessment Plan was amended to not have an additional course, NURS 495: Integrated Clinical Concepts (1). Instead, students who were unsuccessful on Version 2 were required to remediate prior to the next scheduled HESI Comprehensive Exam.

2. HESI Exam Policy for EVOLVE© (HESI) RN Comprehensive Exam

a. The HESI Version 1 Comprehensive Exam will be administered after completion of SBSN 450/450P (Health Alterations III) and SBSN 461/461P Community and Public Health Nursing) and after completion of the NCLEX-RN® Review Course .


c. Students may take HESI Comprehensive Exam 2 for additional practice if they score 900 or greater.

d. Students must repeat HESI Version 2 exit exam with a minimum score of 850. If the student does not achieve a minimum score of 850 on Version 2 the student is required to purchase and take HESI Comprehensive Exam 3.


f. If students are unsuccessful on the first or second HESI Comprehensive Exam, they will be required to remediate prior to the next scheduled HESI.

3. Continue to use concept mapping. 4. Consult with Loretta Manning and Lydia Zager to review and analyze examinations in all clinical courses to

ensure they are meeting the identified content of the NCLEX. 5. Integrate simulation intentionally with structured scenarios for meaningful evaluation of student performance. 6. Meet with students during advisement to review individualized HESI remediation plans. 7. SBSN 308: Pathophysiological Topics in Health Care or SBIO 370: Pathophysiology added fall 2012 as a

requirement for the upper division to help students increase their understanding of disease processes, increase their success in subsequent courses in the program, and increase student performance on HESI tests and the NCLEX – RN (National Council Licensure Exam).

8. Combine didactic and practicum courses to assist students with better integration of the content and application of skills.


P&OD 2.11.10 46



The action plan for 2010 – 2012 was evaluated as follows:

Implement a mandatory remediation program for students at each level by fall 2011. Remediation will be assigned to course faculty • If a student scores less than 850 on a HESI exam for any specialty course remediation is required. • Students must complete the remediation package provided by Evolve on content missed. • Students will be assigned a faculty mentor who will meet with students either collectively or individually for 4 additional weeks to review content missed and practice questions. This action was implemented as scheduled in fall 2011. The results as noted in the above reveal that this strategy was effective. The number of students scoring below 850, completing remediation, and passing the NCLEX exam on the first attempt increased indicating the remediation program was an effective strategy. Endorse concept mapping as additional available faculty teaching strategy. This was endorsed in the May faculty organization meeting in Spring 2011 and implemented in fall 2011. The faculty reaffirmed the use of concept mapping as a teaching strategy and for student clinical planning of care in spring 2012. Offer faculty development programs on test writing in fall 2011 • Refer to Appendix B for the Faculty Development Programs • Focus programs on improving teaching strategies and content of the nursing process Provide course coordinators and senior faculty with Remediation Training for Students during Summer 2011 and Fall 2011. This action was completed by fall 2011. Define specific student learning outcomes and measures for nursing care through simulation scenarios. Individual instructors in clinical courses post their simulations, identifying learning outcomes for the scenarios. The students move through the simulation experience, pre-preparatory work, scenario experience, and debriefing. The simulation staff assists them in meeting their goals for their students. The MBSON is a member of Healthcare Simulation of South Carolina (HSSC) which supports and trains faculty in implementing and enhancing learning through simulation (see Appendix B for Faculty Development). The School of Nursing receives the HSSC course catalog on a regular basis that includes all courses and scenarios currently offered and/or projected by HSSC. The scenarios are updated monthly. Create Integrated Clinical Concepts Course (1 hour) for all first semester juniors. • Meet once a week for two hours for the first 8 weeks of semester. • The purpose is to support, motivate, and test students, so that each student who graduates from MBSON would have an excellent likelihood of passing the NCLEX-RN. • Course will combine support for psychological issues (anxiety and negative self-talk) with cognitive preparation (test-taking strategies, practice questions, study skills). Critical thinking has been integrated into SBSN 306 rather than having a separate Integrated Clinical Concepts course. Note: There was improvement in the drug calculation test pass rate fall 2012 as a result of strategy of 5 weekly drug calculation problems solved by students each week.


P&OD 2.11.10 47



Evaluation of the modifications to the HESI policy are as follows: Implement the following policy in fall 2011: HESI Exam Policy for Specialty Exams

1. Students must complete case studies in order to take the HESI specialty exam. 2. Students must achieve a minimum score of 850 on the HESI exam. If the student does not

achieve a minimum score of 850, the student will receive a grade of Incomplete for the course until the student:

a. successfully completes the HESI remediation and b. remediates with course faculty and c. repeats Version 2 HESI exam

3. The HESI exam grade will be the grade achieved on the last HESI exam taken. 4. If the score is less than 850 on Version 2 HESI and the student achieves a course grade of 81 or

less, the student will not be allowed to progress in the nursing program. This was implemented in fall 2011. The students who scored below 850 usually scored lower on each HESI exam. As a result the student was limited in the time spent on each subject. Item 3 was changed to the highest grade achieved on either of the two exams. This could possibly result in cursory remediation to get the required content covered and calculating the score needed to pass the course may have affected the effort to improve the score on the second exam. HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam

1. The HESI Version 1 exit exam will be administered during week 4 of the semester. 2. Students must achieve a minimum score of 900 on the HESI exam. If the student does not

achieve a minimum score of 900, the student must: 3. Enroll in NURS 495: Integrated Clinical Concepts (1). 4. Repeat HESI Version 2 exit exam with a minimum score of 850. If the student does not achieve a

minimum score of 850 the student must: a. Remediate with course faculty and b. Repeat Version 3 HESI exam at the student’s own expense.

5. If the student earns a score of 850 or greater on Version 2 or 3 the exam grade will be the grade achieved on the last HESI exam taken.

6. If the student earns a score of less than 850 the student will receive a “0” as the exam grade.

The HESI Exam Policy for EVOLVE© (HESI) RN Exit Exam was changed in fall 2011 from being given in week 4 following the 10-week SBSN 450/450P (Health Alterations III) and SBSN 461/461P (Community & Public Heatlh Nursing) courses as well as after the completion This was implemented in fall 2011. The HESI scores improved for both of the cohorts in Spring 2012 with an average >850.


P&OD 2.11.10 48

APPENDIX A.

STUDENT LEARNING OUTCOMES: BENCHMARKS AND PERFORMANCE INDICATORS

At the conclusion of the BSN program, the graduate will be: 1. A critical thinker who uses clinical judgment and ethical decision making to design/manage/ coordinate nursing care based on evidence and current research, a broad

knowledge base and standards of nursing practice. a. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the EVOLVE© (HESI) RN

Exit Exam *. b. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the critical thinking items

on the EVOLVE© (HESI) RN Exit Exam *. c. Ninety percent of students will obtain a minimum HESI Composite score (HESIScore) 850 on the EVOLVE© (HESI) RN Exit Exam * . d. Eighty-five percent of RN to BSN Track students will achieve at least a grade of B (85) on the practicum notebook for the Community and Public Health Nursing

clinical course.

2. An effective communicator who uses various modalities to provide caring, competent, and holistic nursing care to a diverse population across the lifespan.

a. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the EVOLVE© (HESI) RN Exit Exam *.

b. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the therapeutic communication items on the EVOLVE© (HESI) RN Exit Exam *.

c. Ninety percent of students will obtain a minimum HESI Composite score (HESIScore) of 850 on the EVOLVE© (HESI) RN Exit Exam *. d. Eighty-five percent of RN-BSN students will achieve at least a grade of B (85) on the formal presentation in the Community and Public Health Nursing course.

3. A competent provider of nursing care who delivers safe, holistic, therapeutic interventions to individuals, families, and communities in a variety of clinical settings

a. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the EVOLVE© (HESI) RN Exit Exam *.

b. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the therapeutic nursing interventions items on the EVOLVE© (HESI) RN Exit Exam *.

c. Ninety percent of students will obtain a minimum HESI Composite score (HESIScore) of 850 on the EVOLVE© (HESI) RN Exit Exam *. d. Eighty-five percent of RN-BSN students will achieve at least a grade of B (85) on the online ethics presentation in the course, Issues in Professional Nursing

Practice


P&OD 2.11.10 49

4. A professional role model who assumes responsibility and accountability for personal and professional behaviors, ethical practice, and client advocacy. a. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the EVOLVE© (HESI) RN

Exit Exam *. b. The student aggregate will obtain a minimum HESI Composite score (HESIScore) at or above the National Mean Composite score on the role development items

on the EVOLVE© (HESI) RN Exit Exam *. c. Ninety percent of students will obtain a minimum HESI Composite score (HESIScore) of 850 on the EVOLVE© (HESI) RN Exit Exam *. d. 100% of the 4-year and RN-BSN track students will obtain a satisfactory rating on the final clinical evaluation tool.

* The HESI score reflects application of the HESI Predictability Model (HPM) to the composite score. Research studies have found the HPM to be highly accurate in

predicting NCLEX-RN® EXAMINATION success. HESI scores range from 0 to over 1,000, and can be as high as 1,500 (depending on the difficulty level of the exam). Reach suggests 900 as a recommended score and 850 as an acceptable score. Students are provided with a "conversion score" defined as a weighted percentage score that considers the average difficulty of the exam and the average difficulty of the test items answered correctly. It is set on a traditional 0-100 scale. The percentile ranking measures the level of performance as compared to the performance of a national standard or norm group.


P&OD 2.11.10 50

APPENDIX B

FACULTY DEVELOPMENT PLAN EXAMPLES

Spring 2011: Train the Trainer Series Presenter: Deborah Charnley, PhD, RN through the Joint Center for Nursing Research and Scholarship (JCNRS) Session I Role Transition - Clinical Expert to Educator Session II Teaching Behaviours that Facilitate Learning Session III Patient Simulation Technology Design & Utilization Session IV Philosophy and Outcomes of Clinical Teaching Session V Ethical and Legal Issues in Clinical Teaching Session VI Assessment of Critical Thinking & Clinical Reasoning in Clinical Practice

Summer 2011 = Teaching Online Course for Initial Online Teaching Certification - Presented online by: Cindy Jennings

1. Best Practices in Online Course Design Video: Planning an Online Course

2. Managing an Online Course Folder: Course Design Folder: Assessment of Learning

3. Tools for Teaching Online Folder: Teaching Strategies, Presentations, Media Folder: Community-Building & Communication Folder: Blackboard Tools Item: Teaching with Wikis and Blogs Screencast & PowerPoint file

4. Distance Learning Library Support 5. Universal Design and Accessibility for Learning 6. Copyright Considerations

August 17, 2011 = Margaret Hindman, PhD, RN, presented “Simulation.” This included:

• Acquainting faculty to the simulation resources available • Discussing the importance of preparing and following lesson plans with defined student learning outcomes

August 22, 2011 = Jim DiMartino presented, “The HESI Remediation Plan” students receive after each EVOLVE (HESI) Exam. This included:

• A review of the questions missed and rationale - available for students immediately following the exam. Students must complete this review prior to exiting the website. • A review of the remediation plan that is individualized to the students specific needs • Time for faculty to practice the remediation process and interpretation of the student and student aggregate results

September 26, 2011 = Test Analysis Presenter: Margaret Hindman, PhD, RN October 24, 2011 = Classroom Teaching Strategies

http://jcnrsttt.wikispaces.com/Session+I+Role+Transition+-+Clinical+Expert+to+Educator

http://jcnrsttt.wikispaces.com/Session+II+Teaching+Behaviours+that+Facilitate+Learning

http://jcnrsttt.wikispaces.com/Session+III+Patient+Simulation+Technology+Design+%26+Utilization

http://jcnrsttt.wikispaces.com/Session+IV+Philosophy+and+Outcomes+of+Clinical+Teaching

http://jcnrsttt.wikispaces.com/Session+V+Ethical+and+Legal+Issues+in+Clinical+Teaching

http://jcnrsttt.wikispaces.com/Session+VI+Assessment+of+Critical+Thinking+%26+Clinical+Reasoning+in+Clinical+Practice

https://blackboard.sc.edu/webapps/blackboard/content/listContent.jsp?course_id=_751961_1&content_id=_5546860_1







P&OD 2.11.10 51

November 14, 2011 = Nurse Squared (A clinical documentation system) Orientation by Elsevier August 16, 2012 Review, Revise, and Re-Energize Faculty: How to Make Test Questions Easy by Loretta Manning and Lydia Zager on strategies to improve NCLEX scores August 22, 2012 Simulation staff from the Medical University of South Carolina, representing Healthcare Simulation of South Carolina (HSSC) provided simulation training to

faculty to implement and enhance simulation experiences for students

Documents

Academic Program : Assessment Report 2011-2012 Mary Black School of Nursing€¦ · · 2017-09-01Academic Program : Assessment Report 2011-2012 ... a competent provider of nursing