Assuring Data Quality Assuring Data Quality

Dept. of Healthcare-Associated Infection & Antimicrobial Resistance,Health Protection Agency

Jennie WilsonProgramme Leader – SSI Surveillance

0

1

2

3

4

5

6

7

AT BE DE ES FI FR HU LT NL NO UE UN US UW

1164 544 5987 379 3896 4775 1203 473 6011 1007 39678 3933 8462 2098

N=6 N=6 N=54 N=5 N=6 N=200 N=7 N=6 N=33 N=20 N=172 NA NA NA

N in

-hospita

l SS

I/1000 p

ost-

op. pt.-d

ays

0.0%

1.0%

2.0%

3.0%

4.0%

5.0%

6.0%

7.0%

8.0%

AT BE DE ES FI FR HU LT NL NO PL UE UN US UW

1166 544 30478 379 6103 4844 1203 474 6081 1009 1325 39684 3941 8764 2250

N=6 N=6 N=100 N=5 N=12 N=200 N=7 N=6 N=34 N=20 N=17 N=172 NA NA NA

SS

I cum

ulat

ive

inci

denc

e (%

)

2005 Hip prosthesis: inter-country rate (incidence density)

2005 Hip prosthesis: inter-country rate (cumulative incidence)

External benchmarksExternal benchmarks

External benchmarks are a powerful driver for effecting change, but require

standardised data collection methods

standardised analysis

high data quality

central co-ordination

Gaynes 1997

Why is data quality so Why is data quality so important locally?important locally?

Do you know whether action is required?• real problems?• poorly collected data?

0

2

4

6

8

10

Apr-Jun 2004 Jul-Sep 2004 Oct-Dec 2004 Jan-Mar 2005 Apr-Jun 2005P e rio d

SSI SurveillanceSSI SurveillanceBasic methodologyBasic methodology

Targeted at categories of clinically similar

operative procedures

Data collection form completed for each relevant

operation (denominator)

Systematic (active) surveillance after each

operation to detect infections (numerator)

Methods of identifying patientsMethods of identifying patientswith SSI (numerator)with SSI (numerator)

Active

Designated, trained personnel, use a variety of data sources to determine whether an HAI has occurred

Sensitivity = 85-100%

Passive

HAI identified and reported by people other than designated, trained personnel. Requires fewer people but unreliable, definition not applied consistently

Sensitivity: 14-34%(Perl, 1998)

Surveillance methods:Surveillance methods:Sensitivity of case findingSensitivity of case finding

Lab-based phoneSensitivity 36%1.2hrs / 100 beds / week

Temperature / treatment chartSensitivity 65%6.5 hours / 100 beds / week

Lab-based, ward liaisonSensitivity 76%6.4 hours / 100 beds / week

Glenister et al 1992Glenister et al 1992

Systematic surveillance for SSI Systematic surveillance for SSI Lab-based ward liaisonLab-based ward liaison

1. Visit ward/patient 3 times per week– discuss patients with ward staff

– check medical / nursing records

– check temperature / treatment charts

2. Review microbiology reports daily– identify positive surgical site reports

Definitions of surgical site Definitions of surgical site infection (CDC)infection (CDC)Superficial incisional

• involves only skin or subcutaneous tissue• occurs within 30 days of surgery

Deep incisional

• involves fascial or muscle layers• occurs within 30 days, implants within 1

year

Organ/space

• part of anatomy opened / manipulated • infection appears related to surgery• occurs within 30 days, implants within 1

year

Superficial Incisional InfectionSuperficial Incisional Infection

Must meet one of the following criteria:

1. Purulent drainage from superficial incision

2. Culture of organisms from fluid/tissue

3. At least 1 symptom of inflammation (pain, tenderness, localised swelling, redness, heat) and incision deliberately opened to manage infection

4. Clinicians diagnosis of superficial SSI

Deep Incisional InfectionDeep Incisional Infection

Must meet one of the following criteria:

1. Purulent drainage from deep incision

2. Deep incision dehisces / deliberately opened and patient has 1 symptom : fever, localised pain/tenderness

3. Abscess / other evidence of infection in deep incision: re-operation / histopathology /

radiology

4. Clinicians diagnosis of deep infection

Identifying SSIIdentifying SSI

Review patients systematically whilst they are in hospital

Do not rely on reviewing case-notes after discharge to find SSIs

If a patient is prescribed antibiotics do not assume these are for SSI – check with clinician

Check significance of positive microbiology cultures

Make sure any SSI identified post-discharge also meets the definition

Is this an SSI…….?Is this an SSI…….?

Nursing record states:

‘Wound oozing ++ from small lower section. Pressure dressing applied’

Oozing what:

•Clear (serous) fluid, blood, pus?

What was the condition of the suture line?

•Red, swollen, dehisced

Was a wound swab taken, if so why?

Criteria for SSI checklistCriteria for SSI checklistWeblink data entry (SSISS)Weblink data entry (SSISS)

Validation studiesValidation studies

Mannien et al 2007: PREZIES, Netherlands

• Reviewed 859 medical charts; 149 SSI

• Validation team = ‘gold standard’

• PPV = 0.97; NPV = 0.99

McCoubrey et al 2005: SSI surveillance, Scotland• 91 SSI reported validated by case note review

• 10/27 hospitals criteria not recorded

• PPV 94.6% (95%CL 87.9 – 98.2); NPV 99.4 (95% CL 98.3 – 99.9)

(assuming not recorded data valid)

NNIS SSI ‘Risk Index’NNIS SSI ‘Risk Index’

Each operation is scored, and results stratified, using 3 major

risk factors associated with SSI*:

• ASA pre-operative assessment score

• Wound class

• Duration of surgery (T time)

Score between 0 and 3

*Culver et al (1991)

Risk Index factorsRisk Index factors

ASA classification of physical illness

1: normal healthy patient

2: mild systemic disease

3: severe systemic disease

4: incapacitating systemic disease

5: moribund, little chance of survival

Wound classification

Clean: no signs of infection, no body ‘tracts’

Clean-contaminated: body tract entered

Contaminated: spillage form GIT, inflammation, open trauma

Dirty: pus, perforated viscera, delayed open trauma, faecal contamination

Changed by pre-op and intra-op events

T timeT timeassociation between p value and cut point for duration association between p value and cut point for duration of operation (abdominal hysterectomy)of operation (abdominal hysterectomy)

Leong et al 2006

Trend in rate Trend in rate of SSI by Risk of SSI by Risk index groupindex group

0

5

10

15

20

25

30

35

% in

fec

ted

0 1 2 3 u/k All

Risk Index Group

Vascular surgery

0

5

10

15

20

25

% in

fec

ted

0 1 2 3 u/k All

Risk Index Group

Large bowel surgery

Effect of indirect standardisation on Effect of indirect standardisation on crude rates of SSI (vascular surgery)crude rates of SSI (vascular surgery)

Rate of SWI (%)

crude adjusted

0

5

10

15

20

0

5

10

15

20

25

% o

pera

tions

infe

cte

d

Abdom

ial h

yste

rect

omy

Bile d

uct,

liver

or p

ancr

eatic

surg

ery

Coron

ary A

rtery

Byp

ass

Graft

Gas

tric s

urge

ry

Total

hip re

plac

emen

t

Hip h

emiar

thro

plasty

Knee

repla

cem

ent

Larg

e bo

wel su

rger

y

Lim

b am

puta

tion

Ope

n re

ducti

on o

f fra

cture

Small

bow

el su

rger

y

Vascu

lar s

urge

ry

Percentiles90th

75th

50th

25th

10th

Distribution of the incidence of surgical site infectionby category of surgical procedures

Source: SSI Surveillance Service, CDSC

October 1997 to December 2003

Crude rates of SWI for Crude rates of SWI for vascular surgery (95% vascular surgery (95%

CL) by hospitalCL) by hospital

0

5

10

15

20

25

30

35

40

45

0 5 10 15 20 25 30 35 40

Order

Rat

e o

f S

WI

(%)

Hospital

90th percentile

50th percentile

Data to December 2001

Funnel plots used to account Funnel plots used to account for variation in sample sizefor variation in sample size

Total hip prosthesis, January 2000 – March 2005

05

1015

2025

Cu

mul

ativ

e in

cide

nce

0 1000 2000 3000 4000number of operations

95% CI 99% CI

Hospital

Cumulative incidence

2.01

0.65

4.05

1.24

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5

Open reductionlong bone fracture

Knee prosthesis

Hiphemiarthroplasty

Total hipprosthesis

% infected

1.67

0.76

2.29

1.36

0.0 0.5 1.0 1.5 2.0 2.5

Open reduction of long bone fracture

Knee prosthesis

Hip hemiarthroplasty

Total hip prothesis

Incidence density per 1000 days

Cumulative incidenceCumulative incidence

Incidence densityIncidence density

Funnel plots to adjust for variation in Funnel plots to adjust for variation in sample size and length of post-op staysample size and length of post-op stay

05

1015

Inci

den

ce D

ens

ity

0 10000 20000 30000 40000in-patient post-operative days

95% CI 99% CI

Hospital

Incidence density/ 1000 post-op in-patient days

Length of stay inLength of stay in elective surgery is reducing elective surgery is reducing

02468

10121416

1998 1999 2000 2001 2002 2003 2004

Year

Me

dia

n le

ng

th o

f s

tay

Total hips Total knee Hip hemi

Proportion of SSI detectedProportion of SSI detectedpre & post dischargepre & post discharge

0

10

20

30

40

50

60

Nu

mb

er

of

SS

Is Post discharge

Pre-discharge

Barrett et al 2000

Post-discharge surveillance Post-discharge surveillance studystudy

Post-discharge surveillance method Resources +++ - data collection, informing/contacting patients

General practitioners/district nurses – poor response rate to questionnaire

Patients – better response; +/- reliability

Sensitivity of case-findingactive vs. passive surveillance

reliability

Barrett et al 2000

Response rate to PDS Response rate to PDS patient questionnairespatient questionnaires

n = 615951%

22%27%

No Response

Response No Reminder

Response With Reminder

Barrett et al 2000

Response rate affected by ethnic group and age