Methodological Challenges in Analyzing The Hospital Preparedness Data

As it Relates to Response

Ibrahim Kamara, MS, MPH, Sc.D Torrance Brown, MPH

Program Evaluation Section (PES)State and Local Initiatives TeamOffice of Preparedness & Emergency Operations (OPEO)Office of the Assistant Secretary for Preparedness & Response (ASPR)

June 2011

Outlines/Introduction

BackgroundPurposeCurrent HPP CapabilitiesData Collection ApproachAnalytic ApproachResults of Selected SnapshotsMethodological IssuesRecommendationsNext Steps

Learning Objectives

Understand the status of the HPP data assessment and its response capabilities

Obtain a better understanding of the HPP data collection, analytic approaches, and results of selected snapshots

Understand some of the obvious methodological challenges of the data as it relates to response

Obtain a better understanding of the next steps

Background & Purpose

The goal of the Hospital Preparedness Program (HPP) is to ensure awardees use cooperative agreement funds to maintain, refine, and to the extent enhance the capacities and capabilities of their healthcare entities for exercising and improving all-hazards preparedness plans, including pandemic influenza.

Purpose Background Mandatory Formula Awards

• State/Territory Departments of Public Health and Directly funded metro areas (LA, Chicago, NYC, & DC); 62 Awardees

Authorizing Legislation• Established in 2002 under the

Public Health Security & Bioterrorism Preparedness & response Act of 2002

• Pandemic and All-Hazards Preparedness Act (PAHPA) signed in December 2006.

Program Foci• Capacity 2002-2006• Capabilities 2007-Present

HPP Capabilites

HPP Overarching Requirements National Incident Management System (NIMS) Education and Preparedness Training Exercises, Evaluation, and Corrective Actions Needs of At-Risk Populations

Level 1 Sub-Capabilities Interoperable Communication Systems Tracking of Bed Availability (HAvBED) ESAR-VHP Fatality Management Partnership/Coalition Development

Level 2 Sub-Capabilities Alternate Care Sites Mobile Medical Assets Pharmaceutical Caches Personal Protective Equipment Decontamination

Data Collection

Analytic Approach

Data is analyzed using SAS, Excel, and Geographic Information System (GIS) mapping for graphical depiction

Descriptive Statistics Continuous data~ weighted and un-weighted

means or median, std. errors or Confidence interval (CI)

Discreet data ~ proportions, comparative ratios, percentages and CI to profile proxy indices as “snapshots” for situational awareness

Inferential statistics from varied multiple regression analysis

Graphical distribution of States by Rate of Participating Hospitals, 2009

Results: Selected HPP Data Snapshot(EOY 2007, 2008, 2009)

Results: Selected HPP Data Snapshot(EOY 2007, 2008, 2009)

Results: Selected HPP Data Snapshot(EOY 2007, 2008, 2009)

Results: Selected HPP Data Snapshot(EOY 2007, 2008, 2009)

Results: Selected HPP Data Snapshot(EOY 2007, 2008, 2009)

Results: Selected HPP Data Snapshot(EOY 2007, 2008, 2009)

Results: Selected HPP Data Snapshot(EOY 2007, 2008, 2009)

Results: Selected HPP Data Snapshot(EOY 2007, 2008, 2009)

Methodological Issues/Challenges

Subjectivity Measurement indicators are determined from a listing of indicators used in other models

determined by expert panel process Bias

Differences between awardees’ response on their capabilities Inconsistent interpretation of the indices/measures among awardees Lack of specificity on capabilities definitions

Weighting Lack of weighting techniques for prioritization of indicators

Mathematical nomenclatures/combinations Lack of consistent units in summing indicators relating to item being measured Indicators are unit less and mathematics or arithmetic combinations can be odd The values sometimes do not represent anything outside of the context in which the

situation is being compared Selection of Indicators

Not based on tested theoretical concepts, and an understanding of the relationship of the indicators, and statistical relationships

Data Sources Source of the data used to create indicators that describe the measure were defined through the

availability of existing datasets, rather than data that represents vulnerability

Recommendations

Data should be from objective sources and reasonable accessible

Indicators should be standardized and normalized to make community comparison

Indicators should be thoroughly validated with scientific reasoning or concepts

There should be a consensus agreement on the type, scope, and appropriateness of measures and indicators

The indicators and measures should be applicable to field experiences within an institutional framework

The conceptual framework should require collaborative process to determine measures’ importance

Qualitative techniques such as surveys or case studies should be conducted to assess the reliability of data.

Next Steps: Developmental Measurement

Framework Resilient Healthcare Communities

Community Integration/CQI (After Actions)

Communications (Situation Awareness)

Safety & Security Patient Tracking Fatality Management Crisis Standard of Care

Surge Beds Exercises Staff Training NIMS Emergency

Management Procedures & Planning

Coalition Governance

Countermeasures, Supplies & PPE (CDC Grant Alignment)

Medical Evacuation Shelter in Place Plan Volunteer Management (ESAR-

VHP/MRC) Decontamination Isolation

Goal

Candidate Capabiliti

es

Measurement

Approach

Formative and Summative (Process & Performance Outcomes)

1. How well-prepared are healthcare coalitions to respond to mass casualty and catastrophic events, and then return to normal daily operations?

2. How well prepared are health care organizations to respond to daily stressors and events that challenge resources and then return to normal operations as a result of participating in a healthcare coalition?

Questions

Healthcare Coalitions/Health Care Organizations

Candidate

Measure

s

Pilot Testing, Primary Data Collection, Grant Alignment Coordination, and External Data Sources

In Development (Interview Questionnaire for Healthcare Coalitions, Evidenced Based Measures, Existing Relevant

HPP Measures, and Existing Grant Alignment

Outcome

s

Unit of Analysi

s