1 The Medical Informatics Research Center, Ben Gurion University, Beer Sheva, Israel

The Importance of Creating an Ontology-Specific Consensus Before a Markup-Based Specification of Clinical Guidelines

Erez Shalom1, Yuval Shahar1 Eitan Lunenfeld2, Meirav Taieb-Maimon1, Ohad Young1, Guy Bar2, Susana B.Martins3, Laszlo Vaszar3, Yair Liel2, Avi Yarkoni2, Mary K.Goldstein3, Akiva Leibowitz2, and Tal Marom4

1 The Medical Informatics Research Center, Ben Gurion University, Beer Sheva, Israel2 Soroka Medical Center, Ben Gurion University, Beer Sheva, Israel3 Veterans Administration Palo Alto Heath Care System, Palo Alto,CA 4 E.Wolfson Medical Center, Holon, Israel

AI techniques in healthcare: evidence-based guidelines and protocols ; ECAI Workshop 29/8/2006


The Required Infrastructure to support automation of GLs


What are the essential steps for the overall guideline specification process?


A Methodology for GL Specification& for its Evaluation

KE

KE

Before Markup

1.

KE ChooseSpecification

Language

KEsInstructing

the EPs

2.

EP

KE

Choosingtogether GL

forSpecification

EP

KE

3.Creating

together anOntology-Specific

Consensus

4.

SEP

KE

EPs Trainingin the

Markup Tool

5.

EP

KE

Creatingtogether

GoldStandardMarkup

EP

6.

KE

KE

EPs Classifythe GL

EPs PerformMarkup

During Markup

EPEP

7. 8.

•The methodology is embedded within the Digital Guideline Library (DeGeL) architecture•Creating an Ontology-Specific Consensus (OSC) is a crucial, mandatory step before markup

1.


The Importance of an Ontology-Specific Consensus (OSC)

An OSC is a document that describes the core objectives and means of the GL

Refers to the knowledge roles of the chosen (target) specification ontology

Enhances accuracy of the mark-up Decreases variability during mark-up and

during application of the GL


The Consensus-Creation Methodology

Clinical consensus – the pathway of the GL

KE +EP add procedural knowledge semantics to the pathway (e.g parallel order

between the regimens)

Ontology-Specific Consensus

1

2

3

4

KE+EP add declarative knowledge to the pathway (e.g filter condition to the main root of the GL)


The second stage in forming an Ontological consensus


The third stage in forming an Ontological consensus


Results (I): Procedural Complexity of the GLs

Guidelines

Plan Type PID COPD HypoThyrd Total

Cyclic (Periodic) 24 2 2 28If-Then-Else 16 4 6 26

Plan Reference 9 NA 3 12Parallel Plan 11 8 2 21

Sequential Plan 14 3 4 21Simple Plan 28 36 14 78

To be Defined 4 6 NA 10Sum of Plans 106 59 31 196


Results (II): Quality of markups EPs can perform markup! High completeness: 91% of the plans and

97% of the KRs were recreated, compared to Gold Standard markup

Variable correctness, but proportion of perfect scores significantly high (P<0.05)

Several clusters of KRs can be formed by level of difficulty to structure

Creating an OSC is crucial for achieving high quality


Results (III): Types of errorsMarkup Error Rate(MER) per KR, per EP

0.55

0.41

0.12

0.31 0.31

0.09

0.24

0.090.03

0

0.1

0.2

0.3

0.4

0.5

0.6

PID COPD HypoThyrdGLs

ME

R

Total Ontological Clinical

360

199

360

112

360

87

194

79

194

61

194

1898

12

98

9

98

3

2*.

.

i

i

GLinKRsofNo

GLinerrorsofNo= MERi.

The differences in the total numbers of errors between the three GLs were highly significant in a proportion test (P<0.001)


Conclusions o Markup is feasible by EPs!

o The more detailed and structured the OSC was, the lower the total number of errors committed by the EPs for each KR

o The need for a graphical OSC-forming graphical tool


Summary The need for gradual GL specification Creating an ontology-specific consensus

as a first step Using a well defined methodology for the

overall process to increase the quality of the markups

For more information:

[email protected]

http://medinfo.ise.bgu.ac.il/medlab


Questions?


The second stage in forming an Ontological consensus

YESNOis PID

severe? (*)

Patient Treatment and evaluation

Hospitalization anddischarged(sequential)

OutpatientTreatment

Outpatient Treatment andevaluation (parallel)

Outpatient Evaluationand follow up

See 1.2See 1.7

See 1.8

Hospitalization

Post DischargedSee 1.5


Creation of OSC – example:Textual Source in the PID Guideline

Parenteral Regimen A

Cefotetan 2 g IV every 12 hours

OR Cefoxitin 2 g IV every 6 hours PLUS Doxycycline 100 mg orally or IV every 12 hours.

Parenteral Regimen B

Clindamycin 900 mg IV every 8 hours

PLUS Gentamicin loading dose IV or IM (2 mg/kg of body weight) followed by a maintenance dose (1.5 mg/kg) every 8 hours. Single daily dosing may be substituted.

Alternative Parenteral Regimens

Ofloxacin 400 mg IV every 12 hours

OR Levofloxacin 500 mg IV once daily WITH or WITHOUT Metronidazole 500 mg IV every 8 hours OR Ampicillin/Sulbactam 3 g IV every 6 hours PLUS Doxycycline 100 mg orally or IV every 12 hours.


The first stage in forming an Ontology-Specific consensus

Parenteral Regimen A

Cefotetan 2 g IV every 12hours

ORCefoxitin 2 g IV every 6hours

PLUSDoxycycline 100 mg orallyor IV every 12 hours.

Parenteral Regimen B

Clindamycin 900 mg IV every8 hours

PLUSGentamicin loading dose IV orIM (2 mg/kg of body weight)followed by a maintenancedose (1.5 mg/kg) every 8 hours.Single daily dosing may besubstituted.

Alternative Parenteral Regimens

Ofloxacin 400 mg IV every 12hours

ORLevofloxacin 500 mg IV oncedaily

WITH or WITHOUTMetronidazole 500 mg IV every8 hours

ORAmpicillin/Sulbactam 3 g IVevery 6 hours

PLUSDoxycycline 100 mg orally or IVevery 12 hours.

Inpatient treatment -Give one of the following regimens


Results (III): Procedural Complexity of the GLs The Subjective Qualitative Ranking

Aspect ID Aspect Name Aspect Description EP1 EP2 EP3 EP4 Mean STDEV

1 Having Medical expertise All your expertise knowledge regarding the guideline 2 3 3 3 2.75 0.50

2Reading the guideline sources before making ont. Consensus

Reading the textual content of the guideline 2 2 3 3 2.50 0.58

3Knowing the multiple representation level model

The Hybrid Representation Model 3 0 3 3 2.25 1.50

4Understanding Asbru Krs – Procedural part

The clinical pathway described as flow chart 2 2 3 2 2.25 0.50

5Understanding Asbru Krs -Declarative part

Each plan includes some declerative KRs such as Filter Conditions KR

3 2 2 2 2.25 0.50

6 Using of Ontology Key concepts, properties and their relations 1 0 2 3 1.50 1.29

7 Having more than one source The guideline has more then one source 3 3 0 0 1.50 1.73

8 Using DeGeL The Digital electronic Guideline Library 0 0 3 2 1.25 1.50

9 Using URUZ –main interface The tool for guideline markup 1 0 0 1 0.50 0.58

10 Using Plan-body wizard The tool for guideline structuring into tree of plans 1 0 1 0 0.50 0.58

11 Using IndexiGuide The tool for indexing the guideline 0 0 0 2 0.50 1.00

12 Using Vaidurya The tool for searching the guideline 0 0 0 1 0.25 0.50

13 Using Vocabulary serverThe tool for finding terms In standard vocabularies such as Loinc, CPT

0 1 0 0 0.25 0.50

14 Using Spock The tool for Applying the guidelines 0 0 0 0 0.00 0.00

1.29 0.93 1.43 1.57 1.30 0.28 Mean Score


Results (I): The Guidelines

Pelvic Inflammatory Disease (PID) Chronic Obstructive Pulmonary

Disease (COPD) Hypothyroidism (HypoThyrd)

Documents

1 The Medical Informatics Research Center, Ben Gurion University, Beer Sheva, Israel