Upload
kendrick-dunnaway
View
215
Download
1
Tags:
Embed Size (px)
Citation preview
Computer Assisted Evaluation of Clinical Data Quality
Nordic Casemix Conference 4.6.2010
Olafr Steinum, Sequelae ABSeppo Ranta, Datawell Oy
2
Introduction
Reported health care information is widely used by authorities
• For health care planning• For quality analyses• For reimbursement• For science and research
It is of paramount importance that the reported health data are correct and valid
Quality assurance is needed• Coding audits• Automatized controls
3
Datawell DRG QA for data quality assurance
DRG QA is a Datawell product
Uses three different logics for data quality evaluation1. Rules and reference databases (e.g. diagnosis codes) used for NordDRG
grouping2. Evaluation of the order of diagnoses (which one is the primary diagnosis, which
are secondary diagnoses) inferrred from large statistical database (Normalization)
3. Clinical Validation Rulebase (CVRB) created and maintained by Sequelae AB
The software can be used as• part of encoder system for immediate feedback of coding quality• standalone system for evaluation of organization data quality• benchmarking system comparing several peer organizations
4
Datawell DRG QA
DRG QA is a Datawell product ****
Uses three different logics for data quality evaluation1. Rules and reference databases (e.g. diagnosis codes) used for NordDRG
grouping2. Evaluation of the order of diagnoses (which one is the primary diagnosis, which
are secondary diagnoses) inferrred from large statistical database (Normalization)
3. Clinical Validation Rulebase (CVRB) created and maintained by Sequelae AB
The software can be used as• part of encoder system for immediate feedback of coding quality• standalone system for evaluation of organization data quality• benchmarking system comparing several peer organizations
Sequelae AB
Is a joint colloboration between
Emendor Consulting AB, (Staffan Bryngelsson)
and Olafr Steinum (diaQualos AB)
Gunnar Henriksson (DRG Henriksson AB)
Sequelae AB
Is a joint colloboration between
Emendor Consulting AB, (Staffan Bryngelsson)
and Olafr Steinum (diaQualos AB)
Gunnar Henriksson (DRG Henriksson AB)
5
Datawell DRG QA
DRG QA is a Datawell product
Uses three different logics for data quality evaluation1. Rules and reference databases (e.g. diagnosis codes) used for NordDRG
grouping2. Evaluation of the order of diagnoses (which one is the primary diagnosis, which
are secondary diagnoses) inferrred from large statistical database (Normalization)
3. Clinical Validation Rulebase (CVRB) created and maintained by Sequelae AB
The software can be used as• part of encoder system for immediate feedback of coding quality• standalone system for evaluation of organization data quality• benchmarking system comparing several peer organizations
6
PatId Dg-a Dg-d Pr LOS Age Dischg Sex
13213 3 134 HOME M
43242 H10.1 J80 WX101 1 54 HOME N
43242 F0289 E756 GD1BD 6 3 HOSP N
64243 V02.0 2 41 HOME N
34212 O75.7 MAF00 4 34 HOME M
DRG QA – An Example of Indicator Calculation Logic
Age not within acceptable limits
External cause code as principal diagnosis
Input data set
ErroneousICD-10 code
Missing principal diagnosis
PatId Dg-a Dg-d Pr LOS Age Dischg Sex
13213 3 134 HOME M
43242 H10.1 J80 WX101 1 54 HOME N
43242 F0289 E756 GD1BD 6 3 HOSP N
64243 V02.0 2 41 HOME N
34212 O75.7 MAF00 4 34 HOME M
Local procedure code
Mismatch of diagnosis and gender
Validations
7
PatId Dg-a Dg-d Pr LOS Age Dischg Sex FDG FPR SEXDG FAGE
13213 3 134 HOME M 110 0 2
43242 H10.1 J80 WX101 1 54 HOME N 100 110 0 0
43242 F0289 E756 GD1BD 6 3 HOSP N 100 100 0 0
64243 V02.0 2 41 HOME N 120 0 0
34212 O75.7 MAF00 4 34 HOME M 100 100 3 0
DRG QA – An Example of Indicators in DRG QA Database
Errors in Diagnosis coding
Errors in Procedure coding
Mismatch of diagnosis and gender
Errors in Age coding
8
DRG QA Pilot Benchmark Database
Seven Hospital Districts in Finland• DRG QA Database contains patient cases from the Ecomed KPP databases from
2008• Data source: Ecomed KPP used in the 7 hospitals
Three County Councils in Sweden• DRG QA Database contains all patient visits and stays from 2008• Data source: Patient Administrative Systems in corresponding county councils
Number of patient cases• Finland n = 4.928.113• Sweden n = 4.332.206
9
Hospital Districts’Ecomed KPP databases (FI), orsimilar data retrieval from Patient Administrative Systems (SE)
etc.
County Council C
DRG QA Database Formation Process
Datawell DRG QA
ETL
EcomedDRG QADatabase
Ecomed Analyzer
Analysis ofData Quality
ReportingCounty Council B
County Council A
District C
District B
District A
Datawell DRG QAIndicator
Calculation
• Data format transformations: hospital code common code mappings
• Calculation of DRG grouping indicators• DRG normalization• Calculation of CVRB matching
• Includes refence population data (1-year intervals) for standardization
10
Data Analysis for the Current Presentation
For this presentation purporses we produced a sample of the benchmarking database
Rules for formation of the present sample• Included hospital stays (coded as ward stays in the source data)• Length of stay (LOS) over zero days• Excludes psychiatry
After applying above mentioned filters the analysis sample consists of• 493 689 ward stays in seven Finnish Hospital Distrcits’ hospitals, and• 444 255 ward stays in three Swedish County Councils’ hospitals
11
0
20
40
60
80
100
120
140
160
180
200
1 2 3 4 5 6 7 8 9 10
Number of Hospital Stays (LOS > 0 d) per 1.000 standardized population
Female
Male
Background Information: Number of Hospital Stays
Finnish University Hospitals
Non-university (Fi)
Swedish County Councils
n = 493 674 (FI); n = 444 255 (SE)
12
Background Information: Number of Stays in Different Age Groups
n = 493 674 (FI); n = 444 255 (SE)
0,000
100,000
200,000
300,000
400,000
500,000
600,000
Num
ber o
f Sta
ys p
er 1
.000
pop
ulati
on
Ward Stays (LOS > 0 d) per 1.000 standardized population in one-year age groups
Finland
Sweden
13
0
1
2
3
4
5
6
7
1 2 3 4 5 6 7 8 9 10
Day
sAverage LOS
Background information: Average Length of Hospital Stays
Finnish University Hospitals
Non-university (Fi)
Swedish County Councils
n = 493 689 (FI); n = 444 255 (SE)
14
0,00
0,50
1,00
1,50
2,00
2,50
3,00
3,50
1 2 3 4 5 6 7 8 9 10
Avg number of diagnosis codes per ward stay (LOS>0 d)
Background information: Average Number of Diagnosis Codes per Stay
Finnish University Hospitals
Non-university (Fi) Swedish County Councils
n = 493 689 (FI); n = 444 255 (SE)
15
0,00
0,50
1,00
1,50
2,00
2,50
3,00
3,50
1 2 3 4 5 6 7 8 9 10
Avg number of procedure codes per ward stay (LOS>0 d)
Background information: Average Number of Procedure Codes per Stay
Finnish University Hospitals
Non-university (Fi)
Swedish County Councils
n = 493 689 (FI); n = 444 255 (SE)
17
0,000
5,000
10,000
15,000
20,000
25,000
30,000
35,000
40,000
45,000
50,000
1 2 3 4 5 6 7 8 9 10
Number of DRG CC cases per 1.000 population (age and gender standardized)
DRG CC Cases
Finnish University Hospitals
Non-university (Fi)
Swedish County Councils
n = 120 060 (FI); n = 147 740 (SE)
18
0,000
2,000
4,000
6,000
8,000
10,000
12,000
14,000
16,000
1 2 3 4 5 6 7 8 9 10
Number of cases with diabetes code (E10-E14) in any diagnosis position per 1.000 standardized population
Diabetes as Principal or Secondary Diagnosis
Finnish University Hospitals
Non-university (Fi)
Swedish County Councils
n = 20 713 (FI); n = 40 085 (SE)
19
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
1 2 3 4 5 6 7 8 9 10
Average Number of Surgical Procedure Codes per Ward Stay
Number of Surgical Procedure Codes (OR property in NordDRG)
Finnish University Hospitals
Non-university (Fi) Swedish County Councils
n = 493 689 (FI); n = 444 255 (SE)
20
Surgical Procedure Codes after Stays in Operative Wards
n = 283 633 (FI); n = 203 684 (SE)
0
0,2
0,4
0,6
0,8
1
1,2
1,4
1,6
Surgery Gynecology and Obstetrics
Eye Disease ENT
Avg
Num
ber o
f Su
rgic
al P
roce
dure
Cod
es p
er W
ard
Stay
Average Number of Surgical Procedure Codes per Ward Stay (LOS > 0d)
FI 1
FI 2
FI 3
FI 4
FI 5
FI 6
FI 7
SE 3
SE 1
SE 2
21
The classification of diagnosis (ICD-10)
A complex system for collecting data for statistics
• Many axes• Many rules
• Explicit rules• Rules expressed in the Tabular volume in connection to code categories• Rules assumed, but not explicitely expressed
Clinical validation rule base - CVRB• A collection of identified rules
22
Rate of Z51.1 Chemotherapy session as Principal or Secondary diagnosis.Swedish county councils 2008
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Huvuddiagnos
Bidiagnos
Data from Swedish National Patient Registry
Principal dxSecondary dx
23
1 Not to be used2 Ought not to be used
3 Not to be used for children < 15 years4 Ought not to be used for children
< 15 yrs
5 Not to be used in inpatient care6 Ought not to be used in inpatient care
7 Not to be combined with code28 Ought not to be combined with code2
9 Not to be used as principal dx10 Ought not to be used as principal dx
11 Not to be used as secondary dx12 Ought not to be used as secondary dx
13 Not to be used as principal dx in combination with code2
14 Ought not to be used as principal dx in combination with code2
15 Not to be used as secondary dx in combination with principal dx code2
16 Ought not to be used as secondary dx In combination with principal dx code2
17 Not to be used as single code18 Ought not to be used as single code
19 Not to be used in combination with other dagger code than code2
20 Ought not to be used in combination with other dagger code than code2
21 Must be combined with code222 Ought to be combined with code2
23 Must be combined with an external cause code (chapter XX) for toxic agent
24 Ought to be combined with external cause code (chapter XX) for toxic agent
25 Must be used as secondary dx only in combination with principal dx = code2
26 Ought to be used as secondary dx only in combination with principal dx = code2
27 Must be secondary dx when combined with code228 Ought to be secondary dx when in
combination with code2
29 Must be principal dx in combination with code230 Ought to be principal dx in combination with code2
31 Rare code32 Rare code inpatient care33 Rare code outpatient care© Sequelae AB
CVRB Rules 2009
24
Frequency of CVRB violation in a test database from 10 provinces
0,00 %
2,00 %
4,00 %
6,00 %
8,00 %
10,00 %
12,00 %
1 2 3 4 5 6 7 8 9 10
% o
f all
war
d pe
riod
s
Any CVRB Rule violation
n = 493 689 (FI); n = 444 255 (SE)
25
0,000 %
0,001 %
0,010 %
0,100 %
1,000 %
10,000 %
100,000 %
CVRB1 Violations
CVRB2 Violations
CVRB3 Violations
CVRB5 Violations
CVRB7 Violations
CVRB8 Violations
CVRB9 Violations
CVRB10 Violations
CVRB11 ViolationsCVRB13 Violations
CVRB14 Violations
CVRB15 Violations
CVRB16 Violations
CVRB18 Violations
CVRB19 Violations
CVRB21 Violations
CVRB22 Violations
CVRB Violations (Hospital Stays, LOS > 0 d)
Finland
Sweden
CVRB Violation Profile in Finland and Sweden, Ward Stays, LOS > 0 days
n = 493 689 (FI); n = 444 255 (SE)
26
02 0004 0006 0008 000
10 00012 00014 000
CV
RB
1
CV
RB
2
CV
RB
3
CV
RB
4
CV
RB
5
CV
RB
6
CV
RB
7
CV
RB
8
CV
RB
9
CV
RB
10
CV
RB
11
CV
RB
12
CV
RB
13
CV
RB
14
CV
RB
15
CV
RB
16
CV
RB
17
CV
RB
18
CV
RB
19
CV
RB
20
CV
RB
21
CV
RB
22
CV
RB
23
CV
RB
24
CV
RB
25
CV
RB
26
CV
RB
27
CV
RB
28
CV
RB
29
CV
RB
30
CV
RB
31
CV
RB
32
CV
RB
33
Violation Rule
Cas
es w
ith
vio
lati
on
Finland
Sw eden
Distribution of CVRB violation in test database (10 provinces)
CVRB violation rule
27
CVRB Violation rules
Not to be used
Ought not to be used
Not to be used in inpatient care
Not to be combined with code2
Ought not to be combined with code2
Not to be used as principal dx
Ought not to be used as principal dx
Not to be used as secondary dx
Not to be used as principal dx in combination with code2
Ought not to be used as secondary dx in combination with principal dx code2
Ought not to be used as single code
Not to be used in combination with other dagger code than code2
Must be combined with code2
Ought to be combined with code2
Ought to be combined with external cause code (chapter XX) for toxic agent
Ought to be used as secondary dx only in combination with code2
Must be secondary dx when in combination with code2
Rare code
Rare code inpatient care
0 5 000 10 000 15 000
CVRB 1
CVRB 2
CVRB 5
CVRB 7
CVRB 8
CVRB 9
CVRB10
CVRB11
CVRB13
CVRB16
CVRB18
CVRB19
CVRB21
CVRB22
CVRB24
CVRB26
CVRB27
CVRB31
CVRB32
28
CVRB11 Not to be used as secondary dx 273 codes from Ch 21 (Z-codes) only to be used as Reason for admission
CVRB10 Ought not to be used as principal dx 16 code categories, mainly unspecified codes and codes marking sequelae
Example of CVRB Violation rules result in percent of inpatient stays
0,0%
1,0%
2,0%
3,0%
4,0%
5,0%
6,0%
7,0%
8,0%
Finland Sweden
CVRB10 CVRB11
1 2 3 4 5 6 7 8 9 10
29
Information Process and the Identified Sources of Quality Failure
Human-Computer interface
Feeding of structuredinformation into the PAS
Processing rules andlogics of the informationsystems
Usability and maintenance ofnational code systems(ICD, NCSP, DRG etc.)
Current transversal studyof the information process
Entry of data Processing of data
Utilization of dataand information
Code systems
30
Information Process and Benefits of Datawell DRG QA
Immediate feedback ofcoding results to codingpersonnel
Information on organizationdata quality for focusingeducation and other correctiveactions.
Benchmarking data quality withpeer organizations.
Entry of data Processing of data
Utilization of dataand information
Code systems
Reports of data qualityincorporated with otherreporting
31
Data which nobody is using has a quality that nobody
wants
Thank you!