The Measurement and Monitoring of Safety:Drawing together academic evidence and practical experience to produce a framework for safety measurement and monitoringApril 2013

A Report from The Health Foundation based on lessons from the UK’s National Health System

BACKGROUND

• In 2000 the paper, “An Organization with a Memory” shed light on the prevalence of harm, medical errors and safety

• 10 years later, current metrics still cannot accurately illustrate if we are safer today than we were ten years ago

• THIS PAPER is an effort to do just that – to develop a broad framework to measure safety, with three main goals:1. Address differing technical and conceptual issues inherent

to measuring safety2. Understand how metrics can be used to improve safety3. Disseminate this information

• Findings draw on published research, case studies, public data and interviews

DEFINING SAFETY

“The avoidance, prevention and amelioration of adverse outcomes or injuries stemming from the process of healthcare”

• Safety has been previously narrowly focused – need to account for risk, harm, structural components, system reliability and both system and staff resiliency to process deviation (error)

• Safety vs. Quality: Both are deviation from standard processes – safety issues are deviations that result in harm

• Safety issues are also those that are considered “unacceptable” – this emotional label, while helpful in raising expectations, has placed the focus on never events as opposed to systems issues

• Therefore, when measuring must be careful not to separate quality and safety too much – many errors don’t result in harm and harm is often not the result of an error

• Need to look towards likelihood of harm and hazards

DEFINING SAFETY

“The avoidance, prevention and amelioration of adverse outcomes or injuries stemming from the process of healthcare”

• The perils of safety as self-reported events: even when consistently and precisely reported, every person, unit and institution has varying standards for what an event is

• Helpful for self-contained improvement, but not across the board comparison

EXAMPLE: Matching Michigan A study on the sources of CLI’s reproduced by the NHS shows that differing self-reports don’t necessarily come from the desire for better results, but simply the infinite difference in individual perceptions – it is as much a social practice as a technical practice

LESSONS from NHS History• The 2000 report “An organization with a memory” incited an evolution of safety

interventions, measures and reporting similar to that in the US – ranging from public reporting and hospital rating to never events with financial incentives embedded into the delivery system

• The NHS has also developed A Safety Thermometer and Outcomes Framework to measure the prevalence and incidence within four areas: pressure ulcers, falls, urinary tract infections, and venous thromboembolism

BUT, What has been neglected?• Measurement and definition of harm• Behaviors• Risk and Hazard

KEY LESSONS from OTHER INDUSTRIES and SAFETY FRAMEWORKS

• Incident report data should be used as only a corollary to more automated system metrics

• The primary purpose of incident data should be to uncover the systemic hazards in place, at which point these more leading indicators become the primary focus of measurement

• Conceptual safety models echo this approach - events are simply evidence of a more deeply embedded (latent) system problem

• Resilience is a critical factor in the dynamic “state” of safety and can be considered with or separate from broader systems measures that will make themselves apparent through event monitoring

Safety Measuring and Monitoring: Five Classes of Information

Safety Measuring and Monitoring: Five Classes of Information

1) Measurement of Harm: Has patient care been safe in the past?

• both physical and psychological measures• physical harm requires typography: treatment specific,

over-treatment, general harm, failure to provide treatment or missed/delayed diagnosis

• suggestion: global trigger tool

Safety Measuring and Monitoring: Five Classes of Information

2) Reliability: are our clinical systems and processes reliable?

• Both systems and behavior measures

• Hard wiring processes and behaviors – process measures and compliance fall here

Safety Measuring and Monitoring: Five Classes of Information

3) Sensitivity to Operations: Is Care Safe Today?

• The information and capacity to monitor safety on an hourly/day to day basis

• Staff awareness – collective mindfulness• Alert and knowledgeable of changing risk (safety walk-

rounds)• Use Patient Safety interviews, designated officers – these

are structural metrics• Continuous feedback

Safety Measuring and Monitoring: Five Classes of Information

4) Anticipate and Preparedness: Will care be safe in the Future?

• The ability to both anticipate and prepare for problems

• Human reliability analysis or failure modes and effect analysis

• Safety Culture• NOTE – acknowledged this area needs most follow up research

Safety Measuring and Monitoring: Five Classes of Information

5) Integration and Learning: Are we responding and improving?

• The ability to respond to and learn from safety information

• Integrating safety information/data into day to day operations – either through huddles or more automatic processes

Safety Measuring and Monitoring: TEN GUIDING PRINCIPLES1. A single measure of safety is a fantasy2. Safety monitoring is critical and does not receive enough attention3. Anticipation and proactive approaches to safety4. Integration and Learning: invest in technology and data analysis

expertise5. Mapping safety measurement and monitoring across organization6. A blend of externally required metrics and local development7. Clarity of purpose is needed when developing safety measures8. Empowering and devolving responsibility for the development and

monitoring of safety metrics is essential9. Collaboration between regulators and regulated is critical10. Beware of perverse incentives

Questions and Discussion

Link to Full Report:http://www.health.org.uk/publications/the-measurement-and-monitoring-of-safety/