THE EFFECT OF PATIENT SAFETY EFFORTS ON QUALITY OF HEALTH CARE

Fred N. Silverman, Lubin School of Business, Pace University, White Plains, NYJohn C. Carter, Lubin School of Business, Pace University, White Plains, NY

ABSTRACT

Medical errors during hospital stays inflate costs and harm patients. With dramatically increasing health care costs and meager improvements in outcomes in recent years, large numbers of hospitals have instituted programs to improve the quality of care. This paper describes important initiatives to improve the health care safety culture in hospitals and analyzes survey data to evaluate the effectiveness of these efforts. The data do not show significant improvements in patient safety culture over the past six years.

Keywords: Quality Management, Healthcare, Patient Safety

1. INTRODUCTION

The Institute of Medicine wrote in its landmark 1999 publication, “To Err is Human”, that more people died as a result of medical errors (~90,000) than from automobile accidents (43,458).1 The publication of this document led the U.S. Senate Committee on Appropriations to direct the Agency for Healthcare Research and Quality (AHRQ) a division of the U,S. Department of Health and Human Services to lead a nationwide effort to determine the cause of medical errors; use a variety of techniques and approaches to reduce such errors; develop models that minimize the frequency and severity of errors; develop mechanisms that encourage reporting, prompt review, and corrective action; and develop methods to minimize paperwork.2 The AHRQ has spent more than $165 million dollars in 2001 through 2003 on patient safety research and continues funding studies and projects.3

1.1 RESEARCH QUESTIONS

The research questions addressed in this paper are:1. Have the widespread efforts by the AHRQ and hospitals to improve patient safety been

successful over the past six years?2. Since large hospitals - often teaching hospitals with greater financial resources - comprise a

much larger share of the participating hospitals in later years, is there a commensurate improvement in patient safety towards the end of the period studied?

AHRQ has conducted hospital surveys on the culture of patient safety annually from 2007 through 2012. Over this period, an ever larger number of hospitals participated in the surveys and hundreds of thousands of hospital care-givers were asked to respond to questions about patient safety in their institutions. The paper examines the results of these surveys over the past six years to assess the impact of AHRQ initiatives on improving patient safety in hospitals.

2. PATIENT SAFETY INITIATIVES

By 2004, the AHRQ had funded more than 100 studies and projects in four categories to: Identify medical errors and causes of patient injury associated with the delivery of health care. Identify, design, test, and evaluate practices that eliminate medical errors and system-related

risks and hazards compromising patient safety. Disseminate, teach, and implement patient safety best-practices that reduce or prevent actual (or

the potential for) patient injury associated with the delivery of health care. Monitor and evaluate threats to patient safety.4

AHRQ funded demonstration and evaluation projects for identifying threats to patient safety. These initiatives included 16 grants for health system reporting, analysis, and safety improvements; 12 projects

for state, regional, hospital, multi-institutional and ambulatory care reporting systems; and four projects for improving patient safety through the use of electronic medical records.

In order to meet its mission of improving patient safety, some of the major components that AHRQ implemented are:

Create centers of excellence in patient safety research in order to build the knowledge base on the impact of medical errors, identify the root cause of threats to patient safety, study the effectiveness of capturing information on medical errors, and evaluating outcomes of promising interventions.

Establish networks for patient safety research as a mechanism to carry out focused research in quality and health services.

Provide training grants to develop the capacity of new researchers in the field of patient safety.5

More recently AHRQ funded a Quality Indicators Toolkit for Hospitals. Developed by the RAND Corporation and the University Health System Consortium it provides a general guide to using quality improvement methods. The improvement process includes the following steps:

Determining Readiness to Change. Applying QIs to the Hospital Data. Identifying Priorities for Quality Improvement. Implementing Improvements. Monitoring Progress for Sustainable Improvement. Analyzing Return on Investment. Using Other Resources.6

Many additional AHRQ initiatives for reducing medical errors and improving patient safety can be found on their website.7

3. HOSPITAL SURVEY ON PATIENT SAFETY CULTURE

The AHRQ has conducted hospital surveys on patient safety culture every year since 2007 as a way of assessing the culture of safety at individual hospitals. The survey results can be used to raise staff awareness about patient safety, assess the current status of patient safety culture, identify areas in need of improvement, evaluate the cultural impact of patient safety initiatives, and make internal and external comparisons.

The survey asks respondents at hospitals throughout the United States questions on their background, outcome measures at the hospital, unit level safety culture dimensions, and hospital wide safety culture dimensions. For each question, respondents reply to each question on a five point scale whether they Strongly Disagree, Disagree, Neither, Agree, or Strongly Agree to each of 33 statements. They also respond to another nine statements on a five point scale of Never, Rarely, Sometimes, Most of the time, and Always. The respondents give a Patient Safety Grade on a five point scale from Excellent to Failing and categorize the number of safety events they have reported from No events to 21 events or more. Respondents also provide background information on themselves.8 The Hospital Survey questionnaire can be downloaded from the AHRQ website.9 The statements and mean responses are presented in charts below.

In 2007 there were 108,621 respondents from hospital staff in 382 hospitals. By 2012 the number of respondents increased to 567,703 from 1,128 hospitals. Of these, 650 hospitals have reported data from at least two different years allowing hospitals to see changes in responses over time.

3.1 HOSPITAL CHARACTERISTICS

Hospitals participating in the survey came from all geographic regions of the United States. Table 1 shows that the proportion of teaching hospitals grew over the years from 24% to 34%. The geographical distribution of participating hospitals is shown in Table 2. Participation of large hospitals (at least 100 beds) increased from 44% in 2007 to 61% in 2012, as shown in Table 3. The proportions of hospitals participating in the surveys closely matched the proportion of AHA registered hospitals.10

Teaching Status 2007 2008 2009 2010 2011 2012Teaching 24% 26% 31% 32% 34% 34%Non-teaching 76% 74% 69% 68% 66% 66%

Table 1: Proportion of Participating Teaching Hospitals

Region 2007 2008 2009 2010 2011 2012Mid Atl/New England 5% 6% 6% 9% 10% 10%South Atlantic 16% 17% 17% 15% 18% 16%East North Central 26% 22% 27% 23% 25% 25%East South Central 7% 6% 5% 8% 9% 10%West North Central 22% 18% 17% 14% 11% 13%West South Central 8% 7% 7% 11% 11% 12%Mountain 9% 11% 9% 8% 7% 6%Pacific 7% 13% 12% 12% 10% 8%

Table 2: Geographic Distribution of Participating Hospitals

# Beds 2007 2008 2009 2010 2011 20126-24 11% 11% 10% 8% 7% 7%25-49 25% 25% 22% 18% 16% 15%50-99 21% 19% 18% 17% 18% 17%100-199 16% 17% 18% 21% 22% 22%200-299 12% 11% 12% 15% 16% 17%300-399 8% 8% 9% 8% 8% 8%400 or more 8% 9% 12% 12% 13% 14%

Table 3: Distribution of Participating Hospitals by Size

3.2 RESPONDENT CHARACTERISTICS

For most hospitals (90%) surveys were distributed to all staff or a sample of all staff from all work areas. Respondents were able to submit completed surveys either on paper only, online only or both. From 2007 to 2012 the proportion of responses submitted on paper only decreased from 42% to 10%. Administering the survey on paper only had a response rate of 61% while the online or mixed modes had a response rate of 50%. Overall, the response rate remained fairly constant over the years at approximately 53%.

The percentage of survey respondents who had direct interaction with patients was 76% for all years. From 2007 to 2012 the number of participating hospitals increased three-fold, while the number of respondents increased more than five-fold. This may be explained by the fact that a greater proportion of large hospitals participated in the survey. The number of respondents with the corresponding number of hospitals can be seen in Table 4.

Number Participating 2007 2008 2009 2010 2011 2012Respondents 108,621 160,176 196,462 338,607 472,397 567,703Hospitals 382 519 622 885 1,032 1,128

Table 4: Number of Participating Hospitals and Respondents

The Hospital Survey on Patient Safety Culture was given to staff in various work units. The proportion of respondents in different work units is shown in Table 5. As can be seen, the proportions have remained constant over the years. More than 35% of the respondents were registered or practical nurses.11

Staff Position 2007 2008 2009 2010 2011 2012Nurse 36% 36% 36% 36% 35% 35%Other 23% 22% 22% 21% 21% 21%Technician 11% 11% 10% 11% 11% 11%Management 7% 7% 7% 8% 8% 8%Clerk/Secretary 7% 6% 6% 7% 7% 6%Hospital Aide 6% 6% 6% 6% 5% 6%Therapist 5% 5% 5% 5% 5% 5%Physician 4% 5% 4% 5% 6% 6%Pharmacist 2% 2% 2% 2% 2% 2%Dietician 1% 1% 1% 1% 1% 1%

Table 5: Proportion of Respondents by Work Unit

4. DATA ANALYSIS

The data have been summarized by AHRQ by the mean and standard deviation of respondents giving positive responses to each question. A positive response is counted when the response is Strongly Agree or Agree for a positively worded question, or Strongly Disagree or Disagree for a negatively worded question. Thus higher mean percentages reflect better perceptions of performance for all questions. Data were summarized for each of the 42 patient safety culture questions over all six years from 2007 through 2012. In addition, data were compiled by year for the categorized number of medical error events reported by the respondents. The mean and standard deviation of the percentage of respondents reporting No events, 1 to 2 events, 3 to 5 events, 6 to 10 events, 11 to 20 events, and 21 events or more were compiled.

4.1 STATISTICAL SIGNIFICANCE

The survey response data are based on very large samples. For example, in 2007, there were approximately 108,000 respondents in 2007 and 567,000 in 2012. With such large sample sizes, the standard errors of the mean percentage positive responses to the 42 questions are very small. Thus, even small observed changes from one period to another are statistically significant. On any of the questions, an ANOVA test of mean differences over the six years is not significant. Looking at pairwise comparisons between the means for 2007 and 2012, the changes in positive responses ranged from -1% to +7%. As illustrated in Table 6, even a 1% improvement is statistically significant, as illustrated for the following question:

Patient safety is never sacrificed to get more work done

2007 2012 t p-valueMean 63% 64% 27.97 0St. Dev. 11.04% 9.41%n 108,621 567,703

Table 6: Example of t-test of mean response change

Even if the survey results in most cases (35 of 42 questions) reveal statistically significant improvements on the patient safety questions between 2007 and 2012, it is more meaningful to examine the practical significance. Thus the following analysis will answer the research questions on that level.

4.2 MEDICAL ERROR OR SAFETY EVENTS REPORTED

The percentage of respondents reporting medical errors or patient safety events over the years can be seen in Chart 1.

No even

ts

1 to 2 ev

ents

3 to 5 ev

ents

6 to 10 ev

ents

11 to 20 ev

ents

21 even

ts or m

ore0%

10%20%30%40%50%60%

Chart 1: Events Reported in 12 Month Periods

200720082009201020112012

As can be seen from this chart, there is very little change in events reported over the years. The percentage reporting “No events” increased from 52 to 55 percent while the percentage reporting “3 to 5” events decreased from 13 to 12 percent. All other values were the same in 2012 as in 2011. The lack of change in reporting errors can be explained by one or both of the following:

There were no changes in the average reporting of errors in spite of the efforts made by the 650 hospitals that have multiple years of reporting points.

The emphasis on reporting errors increased the reporting percentage but this was offset by the decrease in actual errors occurring.

4.3 PATIENT SAFETY CULTURE REPORTS

There are twelve categories of patient safety culture with three to four detailed questions in each category. Charts for the mean percentage of respondents reporting positive perception for all categories over the years are presented below. As can be seen from Chart 2 through Chart 13, there are very small changes in a few of the averages of culture of safety perceptions from 2007 to 2012. For most questions there are no changes in the means over the years.

In Chart 3 some improvement can be seen in supervisor expectations in promoting patient safety for the first question. In Chart 6 there also appears to be improvement in the overall perceptions of patient safety especially in question 4.

Overall the trends in the averages for perceptions in patient safety are mostly flat. Larger hospitals with more resources have entered the database. In addition, hospitals are participating over a period of years implementing safety initiatives that should raise awareness and improve patient safety.

0%20%40%60%80%

100% Chart 2: Teamwork Within Units

200720082009201020112012

64%66%68%70%72%74%76%78% Chart 3: Supervisor Expectations Promoting Patient Safety

200720082009201020112012

1. Hospital mgmt provides a work climate that promotes

patient safety.

2. The actions of hospital mgmt show that patient safety is a top priority.

3. Hospital mgmt seems interested in patient safety only after an adverse event

happens.

0%10%20%30%40%50%60%70%80%90%

Chart 4: Management Support for Patient Safety

200720082009201020112012

0%

20%

40%

60%

80%

100%Chart 5: Continuous Improvement

200720082009201020112012

1. It is just by chance that more serious

mistakes don't hap-pen around here.

2. Patient safety is never sacrificed to

get more work done.

3. We have patient safety problems in

this unit.

4. Our procedures and systems are good at preventing errors

from happening.

50%55%60%65%70%75%

Chart 6: Overall Perceptions of Patient Safety

200720082009201020112012

1. We are given feedback about changes put into place based on event re-

ports.

2. We are informed about errors that happen in this

unit.

3. In this unit, we discuss ways to prevent errors from

happening again.

0%

20%

40%

60%

80% Chart 7: Feedback About Errors200720082009201020112012

1. Staff will freely speak up if they see something that may

negatively affect patient care.

2. Staff feel free to question the decisions or actions of

those with more authority.

3. Staff are afraid to ask questions when something

does not seem right.

0%10%20%30%40%50%60%70%80%

Chart 8: Communication Openess

200720082009201020112012

1. When a mistake is made, but is caught and corrected before affecting the patient, how often is this reported?

2. When a mistake is made, but has no potential to harm the patient, how often is this

reported?

3. When a mistake is made that could harm the patient,

but does not, how often is this reported?

0%10%20%30%40%50%60%70%80%

Chart 9: Frequency of Events Reported

200720082009201020112012

0%10%20%30%40%50%60%70%80%

Chart 10: Teamwork Across Units

200720082009201020112012

1. We have enough staff to handle the

workload.

2. Staff in this unit work longer hours

than is best for patient care.

3. We use more agency/temporary staff than is best for

patient care.

4. We work in "crisis mode" trying to do

too much, too quickly.

0%10%20%30%40%50%60%70%80%

Chart 11: Staffing

200720082009201020112012

0%10%20%30%40%50%60%

Chart 12: Handoffs and Transitions

200720082009201020112012

0%10%20%30%40%50%60% Chart 13: Nonpunitive Response to Error

200720082009201020112012

5. CONCLUSIONS

Many initiatives have been implemented in the last thirteen years to reduce errors by hospitals, doctors, nurses and other caregivers. The only way to assess the effectiveness of these measures is to obtain objective data on a regular basis and disseminate the results to all stakeholders in improving the quality of health care to foster continuous improvement. The AHRQ survey initiated in 2007 provides a useful step forward in the quest to improve patient safety in American hospitals.

Analysis of the data for the first six years shows some encouraging positive trends, but overall, less than one might expect given the large investments in initiatives to improve patient safety. Likewise, there is no discernible trend upwards in positive responses in the last three years, which is again disappointing given the growing share of large hospitals. In conclusion, the data do not support the premise that the overall culture of patient safety is improving, either through the implementation of AHRQ initiatives or because of the greater resources of larger teaching hospitals.

As the AHRQ survey continues to be administered to an expanding respondent database, the composition will eventually stabilize, making comparisons from year to year more meaningful. In the first few years, the number and type of participating hospitals expanded greatly as did the survey responses from hospital staff members. Once the database stabilizes, it would be useful for AHRQ to publish summary data by hospital type and respondent type to allow easier comparison for self-improvement efforts.

6. REFERENCES

1 Kohn LT, Corrigan JM, Donaldson MS, editors. To Err Is Human: Building a Safer Health System. Washington: National Academy Press; 1999.2 Senate Appropriations Labor, Health and Human Services, Education, and Related Agencies Committee Appropriation Bill, 2001. Report 106-293 (May 12, 2000), p. 195-198.3 AHRQ's Patient Safety Initiative: Building Foundations, Reducing Risk. Interim Report to the Senate Committee on Appropriations. AHRQ Publication No. 04-RG005, December 2003. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/pscongrpt/4 Ibid.5 Ibid.6 AHRQ Quality Indicators™ Toolkit for Hospitals. July 2012. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/qitoolkit/index.html7 http://www.ahrq.gov/qual/patientsafetyix.htm8 Hospital Survey on Patient Safety Culture. June 2012. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/patientsafetyculture/hospsurvindex.htm9 http://www.ahrq.gov/qual/patientsafetyculture/hospscanform.doc10 Ibid.11 Ibid.