0 The Facts Don’t Speak For Themselves: AHRQ 2007 HS Kaplan R Levitan B Rabin Fastman CUMC/NYPH...

1

The Facts Don’t Speak For Themselves:

AHRQ 2007 HS KaplanR LevitanB Rabin FastmanCUMC/NYPH

Getting the Story from Aggregate Data

AHRQ 2007

2

Event Reports as Rumble Strips

• Both can help increase safety by revealing danger

• Neither is reliably quantitative

• Both may create some unwanted noise in the environment

3

Counting: A Means to an End

4

Steinbeck on Counting

5

Focus of Today’s Presentation Discuss real-time database queries using built-in

tools:

• RAI: Risk Assessment Index

• “Single Click” Standard reports

• QBF: Query by Field

• Data Mining: Clustering, CBR, and HAWK –

Constraint - Access: Permissions and roles

6

Access: Permissions and Roles• User-based

– Role within organization– Role within MERS system

• Overarching access rule set – Location– Service line/Department– Employee events/Patient complaints– Type: falls, meds, transfusion, equip, etc.– Read-only

• HIPAA

7

Filter no-harm reports to improve signal-to-Filter no-harm reports to improve signal-to-noise ratio (SNR)noise ratio (SNR)

Risk Assessment Index

8

Two Filters to Enhance SNR

ImpactImpact

Freq

uency

Freq

uency LowLow

HighHigh

MediumMedium

9

“Single Click” Standard Reports

• Ad-hoc reporting in real-time

• MERS has a comprehensive list of reports

10

“Single Click” Standard Reports

11

Medication Events by Specific Type

12

Query By Field (QBF): Exact field matches

QBF’s filtered results can be fed into any report, graph, or spreadsheet

13

Generation of Graphicson data subset using QBF filter

14

Medication Drill-Down: Categories of Ordering Errors

15

Benchmark Against Total and Other Reporting Sites/Hospitals

16

Real-Time, Formatted SpreadsheetsExample: Fall outcomes report, breakdown by unit, opened in Excel

17

User-Customized Spreadsheets

18

Mining Association Rules

Decision Trees

IDClustering

Statistical Clustering

Data Mining

CBR

Similarity Matching

Textual

Numeric

Semantic

Neural Networks

© 2007 by The Trustees of Columbia University in the City of New York.

CBR

19

Why Cluster?• Clusters show us event reports that are similar

across predefined dimensions

• They may represent:– frequency of a type of event– event trends in time – potential prevention, etc

© 2007 by The Trustees of Columbia University in the City of New York.

20

Case-Based Reasoning (CBR)

What is case-based reasoning?• Case-based reasoning is another methodology

for, among other things, identifying clusters of similar events in large databases

© 2007 by The Trustees of Columbia University in the City of New York.

21

Clustering/CBR

Clustering divides large data sets into coherent subsets that can be studied more easily

• Given an event report, CBR will – go through all event reports in database

– compute similarity between them

– find all reports within a certain distance or similarity (defined by the user)

• These reports form a cluster

© 2007 by The Trustees of Columbia University in the City of New York.

22

CBR and Similarity Matching• Using CBR, the computer system can establish

the closest matches to any target event

• It can cluster based on similarity

• It can also identify unique events

© 2007 by The Trustees of Columbia University in the City of New York.

CBR and Similarity Matching

23

HAWK• MERS’ similarity function, HAWK,

uses a vector of pre-assigned weights that corresponds to the vector of variables in an event report record.

• HAWK provides information that can be used to evaluate trends

24

CBR: Another Use

• CBR can be extended to provide solutions to problems based on past experiences in the database– e.g., a help desk

25

The Facts Don’t Speak For Themselves:

“Knowledge resides in the user and not in the collection.”

C. West Churchman in The Design of Inquiring Systems