Introduction Method Experiments Conclusions
Towards the Automated Calculation ofClinical Quality Indicators
AIME’11 KR4HC
Kathrin Dentler, Annette ten Teije, Ronald Cornet andNicolette de Keizer
July 6, 2011
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Outline
Introduction
Method
Experiments
Conclusions
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Quality Indicators
Are tools to “measure” the quality of delivered care in order to
I monitor and improve quality
I help patients to make informed choices
Ideally derived from guidelines / evidence-based
Related to:
I Structure: e.g proportion of specialists to other doctors
I Process: e.g. number of examined lymph nodes
I Outcome: e.g. hospital mortality rate
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Current vs. Ideal Situation
I Natural language →decreased validity and comparability
I More and more (obligatory) indicators →manual calculationtoo expensive
Ideally, indicators should be released in an unambiguous,machine-processable, sharable, standard representation &computed automatically
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Idea: Proof of concept study based on simplifying assumptions
Indicators as semantic queries that retrieve patients who fulfilconstraints (eligibility criteria)
I Gradual method to formalise indicators as SPARQL queries
I Encoded patient data and queries + reasoning
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Clinical domain: GIOCA(Gastro-Intestinal Oncology Centre Amsterdam)
I Patient-centred
I Diagnosis and treatment plan within only one day
I Innovative concept →founders are motivated to measure theirperformance
Example process indicator
Numerator: number of patients who had 10 or more lymph nodesexamined after resection of a primary colon carcinoma.
Denominator: number of patients who had lymph nodes examinedafter resection of a primary colon carcinoma.
Exclusion criteria: Previous radiotherapy and recurrent coloncarcinomas
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Exemplary SPARQL QueryPREFIX xsd: <http://www.w3.org/2001/XMLSchema#>PREFIX ehrschema: <http://apdg.net/owl/schema/>PREFIX sct: <http://www.ihtsdo.org/>
SELECT ?patientWHERE {?patient a sct:SCT 116154003 .
}
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Method to formalise quality indicators into SPARQL queries
Numerator: number of patients who had 10 or more lymph nodesexamined after resection of a primary colon carcinoma.
Step 1) Encode relevant concepts from the indicator by conceptsfrom a terminology
?patient a sct:SCT 116154003 .?coloncancer a sct:SCT 93761005 .?colectomy a sct:SCT 23968004 .?lymphnodeexamination a sct:SCT 284427004 .
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Method to formalise quality indicators into SPARQL queries
Step 2) Define the information model
Numerator: number of patients who had 10 or more lymph nodesexamined after resection of a primary colon carcinoma.
?patient ehrschema:hasDisease ?coloncancer .?patient ehrschema:hasProcedure ?colectomy .?colectomy sct:SCT 47429007 ?coloncancer . // SCT 47429007 = associated with?colectomy ehrschema:procedureDate ?colectomydate .?patient ehrschema:hasProcedure ?lymphnodeexamination .?lymphnodeexamination ehrschema:procedureDate ?lymphnodeexaminationdate .?lymphnodeexamination ehrschema:hasNumber ?numberexaminedlymphnodes .
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Method to formalise quality indicators into SPARQL queries
Step 3) Formalise time constraints and temporal relationships(FILTER)
Numerator: number of patients who had 10 or more lymph nodesexamined after resection of a primary colon carcinoma.
FILTER ( ?lymphnodeexaminationdate >”2010-01-01” ˆˆxsd:dateTime )FILTER ( ?lymphnodeexaminationdate <”2011-01-01” ˆˆxsd:dateTime )FILTER ( ?lymphnodeexaminationdate >?colectomydate)
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Method to formalise quality indicators into SPARQL queries
Step 4) Formalise number constraints (FILTER)
Numerator: number of patients who had 10 or more lymph nodesexamined after resection of a primary colon carcinoma.
FILTER ( ?numberexaminedlymphnodes >9 )
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Method to formalise quality indicators into SPARQL queries
Step 5) Formalise truth value constraints (FILTER)
Example from other indicator:?patient ehrschema:dataDeliveredToDSCA ?boolean .FILTER ( ?boolean = true)
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Method to formalise quality indicators into SPARQL queries
Step 6) Formalise in- and exclusion criteria (FILTER)
Exclusion criteria: Previous radiotherapy and recurrent coloncarcinomas
FILTER NOT EXISTS {?radiotherapy a sct:SCT 108290001 .?patient ehrschema:hasProcedure ?radiotherapy .?radiotherapy ehrschema:procedureDate ?radiotherapydate .FILTER ( ?lymphnodeexaminationdate >?radiotherapydate)
}
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Method to formalise quality indicators into SPARQL queries
Step 7) Construct the denominator by removing constraints thatonly aim at the numerator
Numerator: number of patients who had 10 or more lymph nodesexamined after resection of a primary colon carcinoma.
Remove:FILTER ( ?numberexaminedlymphnodes >9 )
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>PREFIX ehrschema: <http://apdg.net/owl/schema/>PREFIX sct: <http://www.ihtsdo.org/>
SELECT ?patientWHERE {
?patient a sct:SCT 116154003 . (Step 1)?coloncancer a sct:SCT 93761005 .?colectomy a sct:SCT 23968004 .?lymphnodeexamination a sct:SCT 284427004 .
?patient ehrschema:hasDisease ?coloncancer . (Step 2)?patient ehrschema:hasProcedure ?colectomy .?colectomy sct:SCT 47429007 ?coloncancer . // SCT 47429007 = associated with?colectomy ehrschema:procedureDate ?colectomydate .?patient ehrschema:hasProcedure ?lymphnodeexamination .?lymphnodeexamination ehrschema:procedureDate ?lymphnodeexaminationdate .?lymphnodeexamination ehrschema:hasNumber ?numberexaminedlymphnodes .
FILTER ( ?lymphnodeexaminationdate >”2010-01-01” ˆˆxsd:dateTime ) (Step 3)FILTER ( ?lymphnodeexaminationdate <”2011-01-01” ˆˆxsd:dateTime )FILTER ( ?lymphnodeexaminationdate >?colectomydate)
FILTER ( ?numberexaminedlymphnodes >9 ) (Step 4; needs to be removed in step 7)
FILTER NOT EXISTS {(Step 6)?radiotherapy a sct:SCT 108290001 .?patient ehrschema:hasProcedure ?radiotherapy .?radiotherapy ehrschema:procedureDate ?radiotherapydate .FILTER ( ?lymphnodeexaminationdate >?radiotherapydate)
}}
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Method to formalise quality indicators into SPARQL queries
Order: Steps 1 and 2 first; 3 - 6 interchangeably; 7 last
1) Encode relevant concepts from the indicator by concepts from aterminology2) Define the information model3) Formalise time constraints and temporal relationships (FILTER)4) Formalise number constraints (FILTER)5) Formalise truth value constraints (FILTER)6) Formalise in- and exclusion criteria (FILTER)7) Construct the denominator by removing constraints that only aim atthe numerator
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Successfully formalised 4 indicators. Experiences:1) One construct not expressible in SPARQL: “number of re-interventionsduring the same admission or during 30 days after the resection (chooselongest interval)”2) High coverage of SNOMED CT with respect to the colorectal cancersurgery domain (only exception: “Transanal Endoscopic Microsurgery(TEM)”)3) High variability in natural language descriptions4) Domain expert indispensable to resolve ambiguities5) Many concepts and filter patterns occur in several indicators and canthus be re-used
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Procedure
Disease
associatedWith
PatienthasProcedure
hasDisease
dateTime
procedureDate
admissionDate /dischargeDate
boolean
dataDeliveredToDSCA
ExaminationLymphNodes
integer
hasNumber
PrimaryColonCancer
rdfs:subClassOfPrimaryRectumCancer
rdfs:subClassOf
SecondaryColonCancer
rdfs:subClassOfSecondaryRectumCancer
rdfs:subClassOf
rdfs:subClassOf
Colectomy
rdfs:subClassOf
Reoperation
rdfs:subClassOf
ResectionRectum
rdfs:subClassOf
Radiotherapy
rdfs:subClassOf
Figure: Simple OWL Schema (Information Model)
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
2010-10-17
2010-10-19
2010-10-18
2010-11-17patient132
Patient
rdf:typedischargeDate
colectomy132reoperation132
hasProcedure
coloncancer132
procedureDate
Colectomy
rdf:typeReoperation
rdf:type
procedureDate
associatedWith
PrimaryColonCancer
type
true
admissionDate
dataDeliveredToDSCA
hasProcedure
associatedWith
hasDisease
Figure: Synthetically Generated Patient
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Reasoning
Anterior resection of rectum
rdfs:subClassOf
Stapled transanal resection of rectum
rdfs:subClassOf
Wedge resection of rectum
rdfs:subClassOf
Resection of rectum
Laparoscopic-assisted anterior resection of rectum
rdfs:subClassOf
I Derived closure of SNOMED CT with reasoner (CB)
I Loaded the closure, the OWL schema and the patient datainto BigOWLIM; openRDF Sesame 2.4 (SPARQL 1.1 queryfeatures)
I Ran 2 queries per indicatorKathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Results
Data Item I1(lymph nodes) I2(DSCA) I3(meeting) I4(reoperation)
numerator 5,449 44,878 17,439 2,713denominator 9,898 49,848 21,807 49,848
percent 55% 90% 80% 0.5%runtime numerator 14.28 25.12 17.74 9.88runtime denominator 15.90 25.71 15.43 41.36
Table: Number of results and runtimes in seconds
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Conclusions
I Presented 7-step method to formalise quality indicators intoSPARQL queries
I Successfully formalised 4 indicators
I Experiences: only one construct not expressible in SPARQL,high coverage of SNOMED CT; variability and ambiguity inthe original descriptions →domain expert indispensable;reusable concepts and filter patterns
I Proof-of-concept based on self-generated patient data:consistent results within acceptable time
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
Future Work
I Extend method:
1. information model (openEHR archetypes)2. intermediary representation (ERGO? Asbru?)3. involve domain experts
I Evaluate method:
1. formalise more indicators2. use real patient data from several clinical sources3. test reproducibility
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators
Introduction Method Experiments Conclusions
DiscussionThank you! Questions? Comments? Ideas?
Kathrin Dentler
Towards the Automated Calculation of Clinical Quality Indicators