View
34
Download
2
Tags:
Embed Size (px)
Citation preview
Evaluating information quality in the detection of pediatric asthma encounters
ANDREW J KNIGHTON, PHD, CPAINSTITUTE FOR HEALTH CARE DELIVERY RESEARCHINTERMOUNTAIN HEALTHCARE
Problem
• Quality is an attribute of data• Meaningful variation in health data quality• Variation in type and scope• Eliminating errors entirely is not feasible• Impact of data quality on decision quality varies
Defining terms
Data:4.21 liters
Information:Mr. X has an FVC of
4.21 liters on 1/1/2015
Knowledge:Mr. X’s falling FVC levels indicate a
problem
Increasing interactions and interrelationships
Incr
easi
ng
com
ple
xit
y
Blum (1986); Nelson and Joos (1989)
The study
• A retrospective cohort design was used to study patterns of asthma readmission at a large urban pediatric hospital using administrative data (Knighton, 2013)
• Variation in defining asthma encounters in the literature and in practice
• Given the pairing of asthma with certain respiratory conditions, we developed an asthma encounter detection algorithm using two criteria (Knighton, 2014):
• Approach: We compared the results of the algorithm against a “gold standard” criteria for asthma encounter detection through medical record review (n=110)
What we found• Improved
detection accuracy from 0.55 [95% CI: 0.46-.064] using primary dx only to 0.83 [0.76-0.90]
• Increased asthma encounters 55%
• Increased bed days 64%
• Increased estimate of the economic impact of asthma 71% from $8.6M to $14.8M
Knighton AJ, et al, Population Health Management, 2014
Figure 1. Receiver operating characteristic (ROC) curve by asthma ICD-9 diagnostic code position (1-5) for a sample of select respiratory encounters (n=110), including 95% CI for both sensitivity and false positive rate.
Implications
Knighton AJ, et al, Population Health Management, 2014
Figure 1. Receiver operating characteristic (ROC) curve by asthma ICD-9 diagnostic code position (1-5) for a sample of select respiratory encounters (n=110), including 95% CI for both sensitivity and false positive rate.
A
Specificity Sensitivity
B C
Case Detection Methods
Linking particular use with case detection
Particular Use ExampleNature of Decision
Required Evidence QualityCase
Detection MethodSensitivity
False Positive
RateAccuracy
Clinical trial research
Identify intervention cases/controls
Patient-level Low to High NoneLow to Medium
A
Observation/ evaluation studies
Retrospective health services research study
Population-level
Medium to High
Low to Medium
Medium to High
A,B
Quality improvement analyses
Improve health services process for asthma care
Process-levelMedium to High
Low to Medium
Medium to High
B,C
Cost-effectiveness analyses
Impact of community intervention
Population-level
HighLow to Medium
Medium to High
B,C
Conclusions
• Multiple documented methods exist today for detecting asthma encounters
• Using ROC analysis to define evidence quality, we can reasonably link particular use categories with appropriate case detection methods
• A terminology familiar to producers and consumers of health data• Categorizing particular use could allow for improved standardization of
measurement• Potential for broad application to other health conditions and particular
uses
References
Blum B. Clinical Information Systems – A Review. West J Med. 1986; 145(12): 791-797.
Knighton AJ, Flood A, Speedie SM, Harmon B, Smith P, Crosby C, et al. Does initial length of stay impact 30-day readmission risk in pediatric asthma patients? J Asthma. 2013; (6): 1–21.
Knighton AJ, Flood A, Harmon B, Smith P, Crosby C, Payne NR. A novel method for detecting inpatient pediatric encounters using administrative data. Population Health Management. August 2014, 17(4): 239-246.
Nelson R, Joos I. On language in nursing: from data to wisdom. PLN Vision. 1989, p6.
Wang R, Strong D. Beyond accuracy: what data quality means to data consumers. Journal of Management Information Systems. 1996; 12(4); 5-34.