Performance improvement in patient safety and risk ... · leadership has provided direction for other provinces to support the program in a similar manner. In December 2007 the Quebec

Salus Global Corporationsalusgc.com

A review of the national data from hospitals participating in the Salus MOREOB obstetrical patient safety program

Dr. J.K. Milne, MD, FRCSC, FSOGC, FACOG President and CEO Salus Global Corporation

Performance improvement in patient safety and risk reduction with participation in the MOREOB Program

salusgc.com

Salus Global Corporation

© 2010 Salus Global Corporation. MOREOB is a registered trademark of Salus Global Corporation. All Rights Reserved

© Copyright 2010, Salus Global Corporation

2

Overview

The MOREOB (Managing Obstetrical Risk Efficiently) program is a comprehensive patient safety, quality improvement, and professional development program for caregivers and administrators in hospital obstetrical units.

The mission of the MOREOB Program is to create a model of care where patient safety is the priority and everyone’s responsibility.

The MOREOB Program was designed to encourage the participation of all individuals involved in patient care within a hospital’s obstetrical care unit. In creating a culture of patient safety, the priority is safer healthcare, which becomes everyone’s responsibility. In this culture there are shared goals and values, and the integration into practice of High Reliability Organization (HRO) principles and communities of practice become the norm.

Participation Growth The pilot phase of the program was launched in Ontario in July 2002 and quickly expanded to include other provinces. This phase included 21 healthcare organizations located in three provinces, comprising 33 hospitals with a total of 2500 participants. The second phase of the program was the national launch, which built on the success of the pilot experience and expanded to include the Western provinces.

A first for the program was achieved in October of 2004, when Alberta’s Minister of Health and Wellness announced support for the implementation of the Program on a provincial basis. This leadership has provided direction for other provinces to support the program in a similar manner. In December 2007 the Quebec Ministry of Health and Social Services (MSSS) agreed to provincially support the MOREOB Program with implementation to begin in 2008. This added 62 new hospitals to the program to be implemented over the next three years. As of July 31, 2010, 211 hospitals have participated in the Program in nine provinces and one territory involving over 10,000 participants.

salusgc.com




3

The program expanded into the United States in the fall of 2005 with St. Joseph’s Hospital Heath Centre in Syracuse New York becoming the first hospital in the U.S. to actively engage in the Program. Newton, Kansas was the second site, beginning the program in July 2006. In 2009 a licensing agreement was signed with the Risk Management and Patient and Safety Institute (RM&PSI) a division of The ProMutual Group of Boston MA to help market and implement the program in the United States. This has resulted in four more hospitals implementing the program bringing the total to six with 277 participants.

salusgc.com




4 Participants

The breakdown of MOREOB participants by professional discipline continues to be consistent within all participating hospitals and reflects the proportion of professional disciplines practicing in the hospitals.

Health care providers %

Nurses 69%

Obstetricians 13%

Family Physicians 10%

Midwives 8%

The program has been implemented in a number of environments including rural and urban and has been implemented in hospitals that provide primary, secondary and tertiary levels of obstetrical care. A goal of MOREOB was to ensure that at least 80% of the healthcare providers in obstetrical care units participate in the activities of the Program. The 80% target was established in order to engage the critical mass necessary to sustain the change to a culture of safety. The program has surpassed the 80% participation rate at all of the sites.

Measuring the Impact on the Work Environment Measuring the impact of the MOREOB Program on the work environment is equivalent to looking at the social infrastructure of an obstetrical unit within the context of a hospital and community. The MOREOB program uses a balanced score card approach. Using an adaptation of the Kirkpatrick-Phillips model of training evaluation, the MOREOB Program employs a four level performance report card using information gathered from the trends in knowledge enhancement, culture change, the annual environmental scan, the MOREOB program’s impact questionnaire filled out by participants in the hospital program, and a compilation of data from on-site focus groups.

The four levels of evaluation are:

Level Method Measures

1

Reaction

- Questionnaire - Focus groups

- Overall satisfaction, perceived value and recommendations - Effectiveness of tools and activities

2

Learning

- Questionnaire - Focus groups - Core knowledge pre- and post test

- Gains in core clinical knowledge - Caregiver confidence - Caregiver competence - Patient safety knowledge

3

Application

- Questionnaire - Focus groups - CAS - Environmental scan

- Culture change - Practice change - Work processes

4

Business

- Questionnaire - Focus groups - Claims statistics

- Hospital accreditation - Patient satisfaction, care giver satisfaction - Liability cost reduction - Work related stress - Worthwhile investment

salusgc.com




5 Evaluation Results The evaluation process has been completed on 105 hospitals involving 5530 participants with the following impact:

Level 1: Reaction Reaction and Planned Action

• 87% of respondents indicated they would recommend the MOREOB Program to others • The following components were rated by participants on a five point scale. The

percentages below are the proportion of health professionals who rated the features as either effective or very effective. The results demonstrate a strong positive reaction to core content, program tools and activities and their high effectiveness in improving patient safety:

Emergency Drills 82%

Workshop OSCE days 87%

Chapter Content 84%

Skills drills 84%

Case Studies 80%

Mnemonics 76%

Level 2: Learning Clinical Core Knowledge A key objective was to provide and measure the ability for all disciplines within the program to achieve the same core knowledge base. The expectation was that an equivalent, cross-professional core knowledge level would provide the initial foundation for building respect and trust, and lead to improved communication.

Using a standard knowledge base assessment tool that is criterion referenced, program participants have been evaluated using the pre- and post-test results after Module1, 2 and 3. The average mark by profession on the pre-test for these participants showed differences in the MOREOB Program’s core content knowledge between nurses, midwives, family physicians and obstetricians. These differences were relatively consistent regardless of the hospital level of obstetrical care (primary, secondary, tertiary). The post-test questions were drawn from the same question bank.

Results to-Date:

• The average score has increased for each profession in all hospital environments (primary, secondary, tertiary) when pre-test and post-tests are compared.

• The pre-test results varied by profession within a range of 19 percent. This range decreased to 8 percent on the first post-test. At post-test three the range between professions had decreased to 4 percent. This comparable knowledge level of the MOREOB Program’s content across all professions and in all hospital environments is a critical component in fostering the increased trust and respect that is essential in improving performance in patient safety in the work place environment.

• Over 91% of hospitals that have completed the program have made the decision to stay engaged with the MOREOB Program by participating in the Advancing with MOREOB

(AwM) Program. Our results show that the participants are maintaining their enhanced knowledge performance obtained by the end of module three of the MOREOB Program.


5 Evaluation Results The evaluation process has been completed on 105 hospitals involving 5530 participants with the following impact:

Level 1: Reaction Reaction and Planned Action

• 87% of respondents indicated they would recommend the MOREOB Program to others • The following components were rated by participants on a five point scale. The

percentages below are the proportion of health professionals who rated the features as either effective or very effective. The results demonstrate a strong positive reaction to core content, program tools and activities and their high effectiveness in improving patient safety:

Emergency Drills 82%

Workshop OSCE days 87%

Chapter Content 84%

Skills drills 84%

Case Studies 80%

Mnemonics 76%

Level 2: Learning Clinical Core Knowledge A key objective was to provide and measure the ability for all disciplines within the program to achieve the same core knowledge base. The expectation was that an equivalent, cross-professional core knowledge level would provide the initial foundation for building respect and trust, and lead to improved communication.

Using a standard knowledge base assessment tool that is criterion referenced, program participants have been evaluated using the pre- and post-test results after Module1, 2 and 3. The average mark by profession on the pre-test for these participants showed differences in the MOREOB Program’s core content knowledge between nurses, midwives, family physicians and obstetricians. These differences were relatively consistent regardless of the hospital level of obstetrical care (primary, secondary, tertiary). The post-test questions were drawn from the same question bank.

Results to-Date:

• The average score has increased for each profession in all hospital environments (primary, secondary, tertiary) when pre-test and post-tests are compared.

• The pre-test results varied by profession within a range of 19 percent. This range decreased to 8 percent on the first post-test. At post-test three the range between professions had decreased to 4 percent. This comparable knowledge level of the MOREOB Program’s content across all professions and in all hospital environments is a critical component in fostering the increased trust and respect that is essential in improving performance in patient safety in the work place environment.

• Over 91% of hospitals that have completed the program have made the decision to stay engaged with the MOREOB Program by participating in the Advancing with MOREOB

(AwM) Program. Our results show that the participants are maintaining their enhanced knowledge performance obtained by the end of module three of the MOREOB Program.

salusgc.com




6

Confidence and Competence

• 72% of respondents indicated that they gained confidence and were more competent in managing day to day low risk situations

• 75% of respondents indicated that they gained confidence and were more competent in managing high risk situations

Key Principles of Patient Safety

• 70% of respondents gained a better understanding of personal learning needs relative to patient safety

• 64% of respondents gained a better understanding HRO principles and their use in daily practice

• 63% of respondents gained useful techniques for enhancing inter-professional teamwork and communication

Level 3: Application Changing the Culture

To objectively measure the success of the MOREOB Program in achieving a change in culture, a robust measurement tool called the Culture Assessment Survey Tool (CAST) was developed to measure culture change in the work environment in participating hospitals.

The CAST is a survey based on six elements considered critical to a robust patient safety environment. Each element is measured by nine evaluative statements. This allows each participant to personally and confidentially identify both strengths and gaps they feel help or impede the building and sustaining of a patient safety culture in their birthing unit. The Culture Assessment Survey (CAS) is done on-line by all individuals participating in the program at the beginning of the program and it is repeated again with each module.

salusgc.com




7 Results to-Date Aggregate data from 179 hospitals participating in the program reveals that 7781 participants have completed Culture Assessment Survey (CAS) 0, 7302 have completed CAS 1, 4396 have completed CAS 2 and 1857 have completed CAS 3. The variation in the total number of participants completing each survey is reflective of where these hospitals are in the implementation of the three modules of the Program.

On a national level the CAS 0 resulted in none of the 211 hospitals having an average score of four or higher in any of the elements. The elements reflecting patient safety as a priority, teamwork, and open communication consistently scored the lowest. As these hospitals progressed through Modules 1, 2 and 3, the repeat CAS surveys showed improvement in all elements. This demonstrates the longer a hospital obstetrical program stayed engaged with MOREOB, the more their patient safety culture improved as measured by these six elements.

Practice and Work Process Changes • 67% of respondents indicated the program had a favourable impact on the consistent

use of protocols

• 66% indicated improved consistency of practice among caregivers

• 60% of respondents indicated that the Program enhanced communication with the patient

• 60% of respondents indicated the program had improved communication among caregivers

• 82% of respondents reported improved management of obstetrical emergencies such as shoulder dystocia and post partum hemorrhage

salusgc.com




8 Level 4: Business Impact

Tracking Liability Claims Information The Healthcare Insurance Reciprocal of Canada (HIROC) has provided claims information for 39 of the hospitals they insure that are participating in the MOREOB Program.

Separating obstetrics from all other services using polynomial trending, they have demonstrated a dramatic reduction trend in liability carrier (hospital) incurred costs for obstetrics pre and post implementation of the MOREOB Program. (Demonstrated by the blue obstetrical polynomial trending curve in the figure below).

Using the same 39 hospitals HIROC has reviewed the total incurred costs for all other health care services, and has observed a slower downward trend in liability carrier (hospital) incurred costs. (Demonstrated by the red polynomial trending curve in the figure below).

This observed dramatic downward polynomial trending curve for the obstetrical services is a positive reduction trend and this observation is important when linked to the increased trends in knowledge enhancement and culture change. This observation supports a fundamental premise of the MOREOB Program; that an increase in knowledge and patient safety culture achieved through the Program translates to better baby outcomes.

salusgc.com




9 Tracking Liability Claims Information

Using this same data set the average cost per claim in the obstetrical services was compared to all other services in those same hospitals. Looking at this from a cost per claim point of view there is a dramatic downward trend in the average obstetrical cost per claim compared to all other services. To determine if the difference in the average cost per claim between the obstetrical services and all other service was significant a t-Test was performed. The resulting p-value of 0.000006439 confirms the difference is statistically significant. (See figure below)

salusgc.com




10

Tracking Liability Claims Information Further analysis from HIROC of this data provides some projections with respect to the expected total incurred costs which includes the identified but not yet recorded (IBNR) costs. Also projected is the total cost, based on an accurate historical annual inflation rate of 7% for infant claims. The projections also demonstrate a substantial difference in the cost per claim between the projected historical cost based on inflation and the actual incurred cost with and without the IBNR. (See Figure below)

Facilitating Accreditation Accreditation Canada formerly called the Canadian Council on Health Services Accreditation (CCHSA) undertook the task of informing their client organizations about the MOREOB Program in their Maternal/Child Services standards and criteria, beginning in 2007. Feedback from hospitals reveals that participation in the MOREOB Program has reduced the organizational process and preparation time prior to the on-site accreditation visit. This translates to a significant cost savings for the hospital.

$0

$10,000,000

$20,000,000

$30,000,000

$40,000,000

$50,000,000

$60,000,000

$70,000,000

2002 2003 2004 2005 2006 2007 2008

Likely Ul4mate Results Without MORE OB

Likely Ul4mate MORE OB Results

Current Incurred Amounts (ex IBNR)

L&D Claims Costs for 39 MOREOB Hospitals (Data as of 31/12/09)

salusgc.com




11

Caregiver Satisfaction and Work Related Stress

Was it a worthwhile investment?

salusgc.com




12

Summary The MOREOB Program has expanded from the 33 pilots in May 2004 to over 200 hospitals in Canada with over 10,000 participants. These results demonstrate the ability of the program to:

• Engage large numbers of participants in a multidisciplinary process to improve patient safety

• Shift existing cultures to a more focused patient safety culture as measured by the CAS tool, with all elements showing improvement.

• Improve the obstetrical core knowledge of participants in all disciplines and in all environments

• Reduce incurred costs of labour and delivery claims in participating hospitals as reported by claims data provided by the Health Insurance Reciprocal of Canada

• Facilitate the hospital accreditation process.

• Facilitate a process for measuring the impact on the work environment

July 31, 2010

200-717 Richmond St. London, ON N6A 1S2


Corporate Head Office:

Salus Global Corporation200-717 Richmond St. 519.640.7333London, ON N6A 1S2 800.766.8301Canada 519.640.5762

moreob.com

salusgc.com

United States Office:

Salus Global US Corp.Suite 131, 90 West Afton Ave. 800.354.1744Yardley, PA 190672 267.390.0555United States

T

l

F

I

F

Documents

Performance improvement in patient safety and risk ... · leadership has provided direction for other provinces to support the program in a similar manner. In December 2007 the Quebec