Date Received

Fall Semester Assessment Report Form DUE October 31st, 2007

BSN NURSING PROGRAM REPORTAssessment Report regarding Graduating Students of Spring 2007

1. Program Information: Program Undergraduate BSN Program

Department School of NursingCollege Division of Health Sciences

Program Assessment Coordinator

Dr. Susan Kowalski

Semester Spring 2007Report

Submitted byDr. Susan Kowalski

Phone/email 895-3404 Susan.Kowalski@unlv.eduDate Submitted October 26, 2007

2 out of 8 BSN Program Outcomes were assessed in this report.

Which outcomes for this program were measured?

How did you measure the outcomes?

What results did you expect?

#1 Utilize the nursing process as a framework for nursing practice incorporating critical thinking skills.

#5. Integrate appropriate standards of care, protocols, research findings, and

Standardized Tests (Educational Resources, Inc. [ERI]) selecting data relating to areas of Nursing Process and Critical Thinking.

NCLEX (National Council Licensure Exam).

Student Exit Survey(Likert Scale)

Selected Items on ERI exams related to standards of care, protocols and safety.

Class group mean will be at or above the North American mean for nursing process areas and critical thinking areas on following ERI Tests: RN Assessment Critical Thinking

First time Pass Rates for Class will be at or greater than the national average.

Outcome # 1 will be rated with mean of 3 or above.

Class mean scores will be at or above North American mean score on selected items from RN Assessment Exam

continuous quality improvement in providing evidence-based nursing care.

Capstone Quality Improvement Project (for a selected problem in hospital setting - graded by criteria set by Level IV Faculty)

Student Exit Survey(Likert Scale)

100% of students will receive at least 35 points out of 40 according to faculty grading criteria (rubric).

Outcome # 5 will be rated with mean of 3 or above

3. Results, conclusions, and discoveries. What are the results of the planned activities listed above? What conclusions or discoveries were made from these results. Describe below or attach to the form.

Results, conclusions, and discoveries (For supplemental data see attachment)

Program Outcome # 1Ability to utilize the nursing process as a framework for nursing practice incorporating critical thinking skills was assessed by comparing the class mean on two standardized tests (Educational Resources, Inc.) with the North American Average.

Test (Composite) Class Mean North American MeanRN Assessment 57 * 59Critical Thinking 77 71

Class composite scores reflect that our students are below the North American mean on the RN Assessment test, and above the norm on the Critical Thinking exam.

When the steps of the Nursing Process are analyzed further two areas of weakness can be identified:

Steps of Nursing Process Class Mean North American MeanAssessment 62 62Analysis 58 54Planning 56 * 60Implementation 61 64Evaluation 49 * 54

An analysis of the Critical Thinking Process reveals that our students’ scores are higher than the North American average in all areas: Critical Thinking Process Class Mean North American MeanPrioritizing/Discriminating 82 75Inferential Reasoning 79 70

2

Interpretive Reasoning 73 66Goal Setting 86 81Application of Knowledge 78 74Evaluating Predicted Outcomes 70 66

NCLEX ResultsThe above exam results are the exam scores of the Graduating Class of Spring 2007 (N=45) who took the exams during their last two semesters. There is a lag time before NCLEX (the national nursing exam for RN licensure) results are available after students graduate. For the purposes of this report, the NCLEX pass-rates of graduates who took the exam between 1/01/07 and 6/30/07 are used. 61 candidates took the exam between 1/01/07 and 6/30/07. 55 passed the exam and 6 failed the exam, for a pass rate of 90.16%. (See Appendix – National Council Licensure Examination for Registered Nurses Quarterly Report).

UNLV graduates compare favorably with U.S.-Educated candidates taking the NCLEX. According to Quarterly Examination Statistics from the National Council of State Boards of Nursing the national pass rates are as follows for Spring 2007: Jan-Mar 80.0% April-June 81.3% July-Sept 79.7%(See Appendix – Quarterly Examination Statistics – National Council of State Boards of Nursing)

Faculty Research StudyIn an effort to understand factors which may contribute to failure of the NCLEX, and the value of using standardized tests to evaluate outcomes and the School of Nursing’s curriculum change, two faculty members conducted a study of 4 classes who graduated during the semesters of Fall, Spring, and Summer of 2005, and Fall 2006. Data was available on ERI exams as well as NCLEX first time pass rates. The School of Nursing had implemented a major curriculum change in Fall 2005 decreasing the program from 5 semesters to 4 semesters. These 4 classes were the first graduates (N= 147) from the new curriculum. The following research questions are pertinent to this report.

1. Will there be a change in NCLEX-RN pass rates following our curriculum revision?There was a significant change in the first attempt NCLEX-RN pass rate following our curriculum revision. The NCLEX-RN pass rate for the study sample increased to 87.8%. The percent change was 8.5% from the past 5-year mean pass rate for our SON (p ≤ 0.000). (See Appendix - Table 3)

2. Are there significant differences in ERI composite scores for graduates who pass NCLEX-RN on their first attempt compared to those who do not?The study found significantly higher composite scores on 13 of the 16 ERI exams for the graduates who passed NCLEX-RN compared to those who did not (p ≤ 0.000 –

3

0.03). Gerontology, Adult Care II (medical-surgical), Therapeutic Communication, Critical Care, and RN Assessment were highly statistically significant (p ≤ 0.000) Composite ERI scores for Pharmacology, Anatomy and Physiology, Mental Health, and Maternal-Child were all higher for the graduates who passed NCLEX-RN. The difference in mean scores did not reach statistical significance when compared to those who did not pass the NCLEX-RN. (See Appendix – Figure 3).

The School of Nursing faculty believes these results indicate that we have made successful improvements in our program which have strengthened first time pass rates on the NCLEX. We believe the NCLEX is an objective indicator of our graduates’ ability to apply the nursing process and apply critical thinking. The use of standardized exams throughout the program (ERI) has assisted students in analyzing and synthesizing nursing knowledge gained through their nursing courses.

Baccalaureate Exit Survey A Baccalaureate Exit Survey was administered to the Spring 2007 graduates. The following results were obtained for Program Outcome #1. 37 graduates completed the exit survey.

How prepared are you to Utilize the nursing process as a framework for your nursing practice?Unprepared 2.7%Somewhat prepared 8.1% Prepared 51.4%Prepared Well 37.8%

Results of the exit survey reflect that 89.2% of the graduates believe they were prepared or prepared well in meeting program outcome #1. Their mean score on the 1-4 point Likert Scale for the class was 3.24.

Program Outcome #5Analysis of the capstone Level IV “Quality Improvement Project” was used to assess if students were able to integrate appropriate standards of care, protocols, research findings, and continuous quality improvement in providing evidence-based nursing care. This project identifies a particular problem in the clinical area and students work in groups (usually pairs) to provide a solution based on research findings and standards of care. It is a written paper which is presented to classmates, and shared with associated clinical agencies. Some projects have been utilized by agencies to provide data for their regulatory reviews.

A rubric is used to grade this capstone project, and the total possible points are 40.The rubric was developed by the faculty member responsible for the course and clearly identifies the components of the project (See Appendix). All students met the criterion (achieving at least 35 points out of 40).

4

ERI Results Selected areas relating to Program Outcome #5 on the RN Assessment (ERI) exam were used for assessment. Comparison with the North American average revealed the following:

Sections on RN Assessment Test Class Average N. American Average Safe, effective care Environment 60* 62 Basic Care and Comfort 59* 61Pharmacological and Parental Therapies 58 57Reduction of Risk Potential 62* 64Safety and Infection Control 67 65

These results indicate our Spring 07 graduating class was close to the mean, but slightly below it on 4 of the above 5 areas.

Baccalaureate Exit SurveyResults of the Baccalaureate Exit Survey indicate that students perceived they met program outcome #5 (mean of 3.05 on 4 point Likert Scale). Responses to the question “How prepared are you tp utilize evidence based practice in building a foundation for nursing care” were:Unprepared 2.7%Somewhat Prepared 13.5%Prepared 59.5%Prepared Well 24.3%

This indicates that at least 83.8 % of the class felt they were prepared or well prepared to utilize evidence based practice in building a foundation for nursing care.

4. Use of Results. What program changes are indicated? How will they be implemented? If none, describe why changes were not needed.

Program Outcome #1:Changes have been made in the ERI testing policy as a result of the assessment process. ERI exams now count for up to 10% of the respective course grade. The faculty changed the ERI Standardized Testing policy in Summer 07. The nursing process and critical thinking in clinical decision making is taught and applied in all clinical courses. Theory courses also contribute to the knowledge needed for application of the nursing process. Faculty reviewed all medical surgical content taught across the four levels of the program to ensure that essential content was being taught. The Academic Affairs Council reviewed all BSN syllabi to ensure course objectives were aligned with level objectives which are based upon the program objectives. A table of the relationship among course, level and program outcomes are now part of the format for each syllabus. An online 8-

5

week review course for the NCLEX offered by the National Council of State Boards of Nursing has been added to the NUR 424 Course, “Transitions to Professional Practice”. Although the faculty is pleased with our improved NCLEX pass rates, there is fluctuation in the percentage of those passing. We would like to maintain a strong pass rate of over 90% with each graduating class.

Program Outcome #5The faculty values the use of simulation for teaching evidence based practice, and for learning of safe protocols. A Simulation Coordinator (an RN, PhD faculty member) has been hired to assist faculty develop scenarios for use with our simulation patient and other simulators in the lab. Although in the past the clinical lab was used primarily for Level I students in the Fundamentals course, it is now being used with higher level students to learn advanced skills. Insertion of N/G tubes, tracheotomy care, EKG interpretation, central line care, Code Blues, etc. can be practiced in the lab before performing these skills and protocols on patients. Equipment has been purchased which enable students participating in scenarios to be seen live to other students in the classroom. The School of Nursing is presently preparing for a future shared skills lab with Nevada State College and the Medical School to be built next to University Medical Center. This facility will offer even more opportunities for our students to learn skills and clinical decision making in a simulated environment.

The capstone Quality Improvement Project required in the 4th semester Leadership course appears to allow the student to synthesize knowledge gained throughout the program. The rubric developed for grading this project has assisted students in the planning phase and implementation of the project.

Surveys to Alumni and Employers have been revised for the BSN program. Instead of mailing them out, the surveys will be distributed to nurse managers in the hospitals. Two to three hospitals will be selected each semester to obtain feedback about our graduates progress. The BSN Exit Survey has been consistently used for the past 4 semesters.

5. Dissemination of results, conclusions, and discoveries. How and with whom were the results shared?

The Nevada State Board of Nursing notifies the School of Nursing of NCLEX results for graduates who are licensed in Nevada. RN licensure is dependent on successfully passing the NCLEX. The Nevada State Board maintains a web-site where all names of licensed nurses are posted. This information is available to the public and is used in accreditation reports by the School

Currently the School of Nursing is accredited by the National League for Nursing Accreditation Commission until 2010. In 2008 we will apply for accreditation from the American Association of Colleges of Nursing Accrediting Commission. Three years of data will be used in the forthcoming accreditation report written by administration and faculty.

6

The Dean of the School of Nursing, as a member of The Southern Nevada Nurse Executive Council, meets regularly with the Chief Nursing Officers of hospitals in Las Vegas. Verbal reports are shared with these nurse leaders of the community.

Each semester nursing faculty meet with the Nursing Education Departments of hospitals and nurses associated with clinical learning experiences with our students. Changes in curriculum are shared.

The Health Science Advisory Newsletter is mailed out to our constituents. This newsletter communicates happenings in the school of nursing.

Information about the programs of the School of Nursing are posted on our website (which was attractively reorganized during the Summer of 2007).

Lastly, students are informed of our NCLEX pass rate in classroom settings.

7

8

9

Appendix

10

Table 3. Demographics: Graduates with Available First Attempt NCLEX Data (n = 147)

Total Available Data(n = 147)

Pass NCLEX (n = 129, 87.8%)

Did Not Pass NCLEX (n = 18, 12.2%)

Age Mean/SD 23±7.79 23±8.07 23.3±4.57Range 18-55 18-55 20-36Mode/Median 21/23 21/23 22/22

# % # % # %Entered SON

Fall 2004 36 24.5 31 24.0 5 27.8Spring 2005 40 27.2 33 25.6 7 38.9Summer 2005 38 25.9 35 27.1 3 16.7Fall 2005 33 22.4 30 23.3 3 16.7

GenderMale 17 11.6 15 11.6 2 11.1Female 130 88.4 114 88.4 16 88.9

Ethnic GroupWhite 76 51.7 69 53.5 7 38.9Asian 46 31.3 39 30.2 7 38.9Hispanic 12 8.2 8 6.2 4 22.2Black or AA 8 5.4 8 6.2 0Native American 2 1.4 2 1.6 0NA 3 2.0 3 2.3 0

Admission GPA 3.00 - 3.24 36 24.5 28 21.7 8 44.43.25 - 3.49 33 22.4 30 23.3 3 16.73.50 - 3.74 34 23.1 28 21.7 6 33.33.75 - 4.00 41 27.9 40 31.0 1 5.6NA 3 2.3 0

SON = School of Nursing; AA = African American; GPA = grade point average; NA = not available. National Pass Rate 2005 - all programs/BSN only = 86.7%/87.3%

Appendix

MS = Medical Surgical; *p < 0.005; ** p ≤ 0.000

11

Figure 3.

Significant Differences in ERI Composite Exam Scores: Graduates who passed NCLEX compared to those who did not pass

Date Received

Rubric for Grading Improvement ProjectStudent (s) (no more than 2) Preceptor:

Clinical Facility and Unit: Preceptor’s title:

Topic Needs Improvement Satisfactory Good Exemplary Possible Points

Student points

Purpose of Paper/Introduction

The purpose of the paper is not clearly stated or does not match the content. If content outline has been provided, paper deviates. (.25)

An attempt is made to state purpose of paper, but may be unclear to reader. Content of the paper frequently digresses from the stated purpose. (.5)

Writing occasionally digresses from the stated purpose. But reader can understand purpose of digression. (.75)

The purpose of the paper is clearly stated and provides an organizational framework. Content consistently attempts to meet purpose of the paper. (1)

1

Section I: Introduction

and selection of

standard

Identification of deficiency.

Identification of Standard. Effect

that deficiency is having on patients

or personnel.

No standard identified.

No information related to effect (.5)

Standard provided But…

No indication of impact on patient care. (1)

A statement related to the agencies policy or procedure was stated.

Spoke to effect in generalized terms. (1.5)

A specific standard from an external organization is sited. (1)

The effect of the discrepancy on patients and/or staff is discussed. (1)

2

Members and rational for

improvement team

membership. Parameters of

decision making.

No mention of improvement team members. No indication of directions from management. (.0)

Names provided, areas of expertise not included.

Stated, but not described. (.15)

Names provided positions and areas of expertise included. (.3)

Names and qualifications of team members provided, as well as rationale for membership on team. (.5)

.5

Section II: Clarify Knowledge related to the Discrepancy.

Characteristics of the

Discrepancy?

Discrepancy partially described effects on personnel or clients not described. Organizational readiness for change not discussed. (.5)

Discussion includes “who”, but does not provide information on all assessment variables. No statement related to organizations readiness for change. (1)

Discussion includes most assessment variables, but has limited statements related to organizational climate or readiness to change. Does not state variables which would prompt or delay change. (1.5)

Identifies who (personnel/client) is involved? (.5) States when the discrepancy began? (.5) States when the discrepancy occurs? (.5) Indicates if the discrepancy is constant or variable? (.5) Identifies how the organizational climate and resources affect the discrepancy? (.5) Discusses situational

3

variables that could influence the organizations ability to change(.5)

Problem definition

The problem defined does not match the discrepancy noted. Problem appears superficial.(.5

Problem is related to the data collected, but seems to be tangential to the summary statement(1)

Problem addresses aspects of the data. Recognition that the problem will not be fully addressed is included (1.5)

Problem definition is based on descriptive data. Student indicates she/he student believes to be the root cause of problem. (1)

1

Section III: Planning

Identify various solutions or

options to meet the

Discrepancy/solve the problem.

Substantiate the option s

identified.

Solutions suggested limited. Modifications to current solutions. Little critical thinking.(1.5)

Solutions are limited. Appear to encompass aspects of trial and error.Limited critical or creative thinking (3).

Solution options indicate critical thinking about the preferred outcome. Are based on opinions and/or experience (nurse manager, student, written articles)(4.5)

Solutions are evidence based. [Research article (4) Best Practice (3) Opinion Based (2)] Variance to suggestions in article discussed(.5) Solutions indicate critical and creative thinking (.5)

5

Option Selection

Limited information provided on why the option was chosen. (1)

Written narrative describing why the option was chosen.(2)

Utilizes a table showing criteria of acceptability, options judged against realistic, meaningful criteria. (3)

Utilizes a table showing criteria of acceptability (2), judged against realistic, meaningful criteria(1) Explanation of grading criteria and rationale for selected option (1)

4

Identify time line and

resources needed to

implement.

Does not identify time line or resource requirements. (.5)

Defines time line, but seems inappropriate to the project described. Partially recognizes resources needed to implement. Not realistic on resource needed. (1)

Appropriate time line. States realistic estimates of resources needed. Identifies possible sources. (1.5)

Completes a time line, using a recognized methodology (Gnnat chart, PERT chart). (1)Accurate estimate of resources needed. Resource source identified with concurrence of availability. (1)

2

Determine how the

improvement will be

evaluated.

States that quantitative and qualitative measures will be developed. No direct relationship to identified problem. (1)

Identifies in global terms the focus of the evaluation strategies. Mentions the identified problem. (2)

Identifies one quantitative data source. Identifies one qualitative data source. Matches the defined problem, but undeveloped. (3)

Provides examples of the both the qualitative (1.5) and quantitative (1.5) data collection tool that will be used to measure the outcome of the project. Measurement matches

4

13

the defined problem. (1)

Identify strategies and

tactics to overcome resistance

Describes strategies, but not one of traditional strategies, or does not appropriate name strategy. (.5)

Describes one of the traditional strategies, but does not identify any specific tactics that will be used. (1)

Describes one of the traditional strategies, and names at least two tactics that will be used to overcome resistance. (1.5)

Describes use of one of the traditional strategies of change (1), and provides description of two tactics used to overcome resistance. (1)

2

Organization

Content does not follow outline. (.25)

The content is not arranged logically. Reader can figure out what the writer probably intends, but not without a great deal of concentration. (.5)

The majority of ideas are arranged logically to support the outline of the paper. The reader can follow the line of reasoning. (.75)

Ideas are arranged logically to support the outline of the paper. Ideas are clearly linked. Reader can follow line of reasoning.(1)

1

Content

Paper does not reflect adequate grasp of content. (.25)

Paper reflects marginal understanding of the content. Incomplete or inaccurate analysis and explanations of concepts. Little attempt is made to tie theory to practice. (.5)

Writer appears to be at ease with the content. Most explanations of concepts are complete and accurate. Some helpful applications are included. (.75)

Writer provides an accurate and complete explanation of key concepts. Writing reflects depth and provides appropriate application of the content (1)

1

Grammar, Spelling, Mechanics

Numerous errors, which obscures the meaning of the content. Writer uses clichés and colloquial language. (.5)

Reader is distracted by awkwardly constructed sentences and grammatical errors. Word choice appears limited. Some words used inappropriately. (1)

Occasional, ineffective writing style, but the errors do not represent a major distraction or obscure the meaning. Sentences are well phrased. Word choice is generally good. (1.5)

The writing is free or almost free of errors. Writer goes beyond the generic word to find a more precise or effective word. Sentences are well phrased and flow smoothly from one to another. (2)

2

Format & Aesthetics

Paper does not have scholarly appearance. Does not fulfill faculty specifications as to length. Format changes throughout, e.g. font type, size, etc. If figures and tables are used, may not be labeled

Paper is typed, neat. Paper has a consistent format. Approximates faculty’s specification as to length. Figures and tables are legible but may not help reader

Paper is typed. Neat and meets faculty specifications as to length. Tables and figures are neatly done and provided intended information. (1)

Paper has professional appearance. Meets faculty specifications as to length. Figures and tables are presented logically and reinforce the text of the paper. (1.5)

1.5

14

appropriately, does not provide intended information to reader. (.25)

understand communication. (.5)

Quality and Use of References

Poor referencing system used. Quality of references is poor. (.5)

Inconsistent or illogical referencing system. Many unsubstantiated statements. Reader is confused about the source of the ideas/content. Quality of references is marginal. (1)

Consistent referencing system. Few unsubstantiated statements. References provide support to content. Quality of references is acceptable. (1.5)

Attribution of content/ideas is clearly and fairly represented. (1) Utilizes high quality, professional sources (1).

2

Most Recent Edition of APA Manual

Format of document is not recognizable as APA (.5)

Format of document reflects incomplete knowledge of APA format.(1))

APA format is used with minor violations. (1.5)

APA format is used accurately and consistently. (2)

2

Section IV:Implementatio

n and Evaluation

Describe the implementation

of the improvement

project

Roles of participants undefined. Effectiveness of time line and resource need identification not complete. (.25)

Describes two out of three components (Roles, time line or resource identification) (.5)

Two out of three components adequately addressed (roles, timeline or resource identification) (.75)

Describes involvement of student and team members. (.5) Evaluates time frame and describes resource utilization. ((.5)

1

Describe the outcome of the

improvement project.

Limited overview of outcome. (1.5)

Sufficient narrative overview of outcome.No data collection forms included. (3)

Adequate narrative description. Included graphic (histogram, praeto chart, etc) to summarize outcome. No data collection forms included. No message for future change agents. No statement of knowledge gained. (4.5)

Appropriately designed graphic and narrative description of outcomes. (2) Explanation of interfering variables if project not completely implemented (.5). Data collection forms attached. (1) Provides prescription for future change agents. (.5)Identified knowledge gained. (1)

5

15