Toward a Guideline Authoring Workbench Session #1 May 6, 1999 Barry G. Silverman, PhD

Toward a Guideline Authoring Workbench

Session #1

May 6, 1999

Barry G. Silverman, PhD

Penn People

Lab for Informatics &

Intelligent Systems Tech

(LIIST)• Barry Silverman, PhD• Chen• Tony Yuan

Penn Database Group• Peter Bunneman, PhD• Val Tannen, PhD• Arnaud Sahuguet

Realtime Systems

Performance Lab• Insup Lee, PhD• Oleg Sikolski, PhD

Outline

• Purpose & Challenges

• Authoring Process1. Protocol Eligibility

2. Wellness/Prevention

3. Chronic (Ambulatory Care) Guidelines

4. Acute Care Guidelines

5. Critical Pathways

6. Education & Decision Support Materials

• Maintenance Requirements

• Summary

Purpose & Challenges

• Purpose of Today’s Session– Hypothetical “Tour” of finished CADSE

• Challenges to be Resolved Today

Challenges for Today

• Expose the Dream • Work Toward a Unified DSS View• Define Guideline Workbench Components

– Breadth

• Analyze Alternative Implementations (Depth)– Make vs. Buy– Usefulness, Scalability, & Usability

• Synthesize a number of Workbench Requirements

Figure 1 - A Proposal to Merge Knowledge Life Cycle Management, Open Standards, and Integrated Environment

Authoring Test & Eval Maintaining

ElicitationWizards

KB Models,Skeletons

Knowl.Grapher

Parser

HL7 RIM,XML/Kona

CommonMedical TermServer

DebuggingAgents

ClarityChecks

WorkabilityTesters

PerformanceEnhancers

CoherenceVerifiers

Correspon-denceValidators

Guideline ProjectMgt Support Sys.

RqmtsMapper

IRM

ChangeControl

Module History,Documentation

Milestone &WBS Viewer

Open-SW Canonical Form Guideline Repository

Rules for Workflow Templates for Notes Alerts/Remdrs for Orders

Data HandlineSemantic Messaging(Publish, Notify, Remind) Presentation

ExecutionEnvironment

zeus.seas.gwu.edu hera.seas.gwu.edu

poseidon.seas.gwu.edu

prometheus.seas.gwu.edu

INTERNET

Representation Repository

Guidelines RepositoryGuidelines Representation Toolkit Server

and Intelligent Surveys Generator

Client (provider/consumer station)

Guidelines KnowledgeProcessors Server

Installed Software:a.- Windows NT Workstation ver 3.1b.- Netscape Fast Track Server ver. 2.1(hostingautomated guidelines before publishing)c.- Sun Servlet Development Kit and RuntimeEnvironment (Servlet Proxy)d.-Guidelines Representation Toolkit andGuidelines Servletse.-Intelligent Surveys Generator Servlets (usingJESS to process questionnaires branching rules)f.- Guidelines Knowledge Processors Client class(connecting to Guidelines Knowledge ProcessorsServer)g- Sandia Labs.' JESS classesh.- Netscape Navigator ver 3.1

128.164.12.2 128.164.12.6

Installed Software:a.- Windows NT Server ver 3.1(running FTP service)b.- Netscape Enterprise Server ver.??(hosting Guidelines Repository andSurveys)c.- Bulletproof Jagg (ODBC/JDBCgateway)d.-Microsoft SQLServer ver 2.51(hosting Representation Repository,Guidelines Directory, & SurveysRepositories)e.- Netscape Navigator ver 3.1 (asenvironment's Client )

Surveys Repository

Installed Software:a.- Windows NT Workstation ver 4.0b.- Guidelines Knowledge ProcessorsServer (listening in port 18000 andreplaying using ports 1800x x=1...n)

Installed Software:a.- Windows NT Workstation ver 4.0b.- Netscape Communicatior 4.x asWWW browser

Outside word

Installed Software:Any browser compatible withJavaScript ver 3.x

Deployment Map

HFHS Vendor Components

Kn. Mgt. WorkbenchEnterprise Guideline Repository

Clinic-SpecificGuidelineRepos.

Division-SpecificGuidelineRepos.

Enter-prise

Local

Clinic-SpecificDivision-Wide

Kn. Administrators

Clinic-Access toKM Toolkit,Guideline Changes,& OLTP System

CADSE Ingredients (ATP Diagram)

G uide line Type(R e-use)

E lig ilityC rite ria

S ke le ta lP lans

R iskA ssessm ent

G LIFS em antics

E xtens ionsto G L IF

O rderingG u ide lines

W ork flowR u les

A rden- & H L7-C om p lian tK now ledgeM ed ia to rs

F eedbackG u ide le ts

SOAPCharting &

Paths

Visual Programming w /Controlled M edical Terminology

Groupware for Distributed Authoring, Checking, Debugging, Adapting, M aintaining

Decision Support Repository

Human Authors, Testers, M aintainers

C ross-A u thor C on flic t R eso lu tion In te r-G u ide line C on trad ic tion C heck ing

KNOWLEDGE MANAGEMENT GRAND CHALLENGE

National repository to capture/share “practice decision rules” and electronic guidelines (open, reusable, machine form)

JIT Filtered Push, Not Pull

NOTE1: Need intuitive editing yet guaranteed reliability for knowledge-ware.

NOTE2: Local user toggles/adapts desired widgets,

NOTE3: The Infrastructure Challenge (NCIS is assumed)

RQ - Guidelines-in-Use (tackling the authoring component)

• Can a terminology-enabled, visual editor/wizard be created that:– Is easy for medical professionals to use (cognitive fit)?– Supports guideline knowledge representations (e.g, eligibilities,

algorithms, rules, decision tables, etc.)– Causes guidelines to be better written (coherence, clarity,

completeness, workability, etc.)– Permits local revising/adapting to prevailing practice– Helps authors produce guidelines that “make the right thing the

easiest thing”

Knowledge Engineering Paradox

• Expert clinicians know what they don’t know, but don’t know what they know.

• Expertise is situationally triggered– Its hard to do a data dump a priori

• We need to spend this time together to “discover” your knowledge of guideline authoring– there ain’t no shortcut -- clinician time is essential to

design CADSE right

Work Toward a Unified DSS View(requires clinicians to specify it)

•What are you trying to put in front of the clinician-author ???????

•What should they put in front of clinician-practitioners ??????

•AGAIN: KE theory says you aren’t able to elaborate this fully a priori (so lets walk through some situations together)

The CADSE “Dream”

A Tour of what it could be.

(with apologies to Asymetrix Multimedia Toolbook Instructor)

Guidelines Representation Toolkit - Prototype Implementation ver. 1.0 alpha

14

Knowledge Management Workbench - Prototype Implementation

ver. 1.0 alpha (CADSE)

For the rest of this talk

Try to picture yourself as a clinician-AUTHOR using the CADSE

workbench

Scenario

Tom, the head of Guideline Development for Best HMO, purchased CADSE in order to develop a suite of guidelines including but not limited to:

1. Protocol Eligibility2. Wellness/Prevention3. Chronic (Ambulatory Care) Guidelines4. Acute Care Guidelines5. Clinical Pathways6. Education & Decision Support Materials

Tom found that CADSE ate up these tasks like a knife through butter.

One thing he particularly liked was CADSE’s breadth as a toolbox:

This interface allowed Tom to quickly complete his range of authoring needs

• Using Templates of Guideline Types

• Selecting Objects From Library (add new ones)

• Setting Objects’ Properties

• Filling Out Object Content

• Adding Behavior (via Script)

• Getting Wizard Help (where needed)

Working with templates

Tom began his guideline authoring using

one of the many templates. A template

is a prebuilt “book” that you customize

by adding your own content. Templates

save time and lend a consistent look and

feel to your guidelines.

In addition to using one of the templates

provided with CADSE, you can create

your own, custom template to distribute

to other guideline authors.

TOP DOWN

GUIDANCE PRINCIPLE

Author your Guideline

After Tom chose a template, he then

added his content. CADSE’s easy-to-

use authoring and development tools

make this task simple.

And CADSE is powerful enough to

handle a wide variety of technical

tasks—from creating and managing

new guideline objects to adapting those

in a reuse library.

PRINCIPLEOF

REUSE

Select objects from CADSE’s Library

He added features to his guideline by

selecting from the wide variety of

objects available in the CADSE

Library.

Library objects have built-in behavior

to handle such tasks as eligibility,

decision tables, workflow and logic

charts, and clinical paths, among

others.

ECOLOGICALARTIFACTS PRINCIPLE

ELICITATION TOOLS CONTAINER(Word processor, WWW browser)

Terminology-Enabled, Visual Guideline Editing Component

GenericTasks Assembly Wizard

Sentence Logic Constructors

Syntax structures

HL7 RIMInterface

GLIF SemanticPrimitives

ARDEN Syntax

Reusable parts libraries interface

Data & knowledge representation & access parts (OO and XML)

Datapath forms

Generic Tasks Parts

•Guidln Qualif•Risk Assmnt•Severity Lev.•Diagnosis•Treatment•Education

Clinician Input Parts

•Parameter tables•Decision tables•Rule structures/MLMs•Intention-based relations•Intermed Guidelines Syntax Structures

Consumer Input Parts

•Patient profile forms•Consumer self-assessment

User Communication

Parts

•email generators•Paging facility•XML generators

GLIF-Electronic Guideline Objects (EGO)and XML/DTDs•ARDEN Curly Braces/Term Server

•HL7 EDI

Semantic & Vocabulary

Servers Interface

Setting object properties

Tom customized objects in his

guideline by setting their properties.

Each object, such as an eligibility

assessment or clinical path, has

properties that define its appearance

and behavior.

Tom could easily set an object’s

properties using the Properties

dialog box or by responding to a

wizard.

POLYMORPHISMPRINCIPLE

(Cmd Keys, DMI or Agents)

Customize guideline behavior without limit using JavaScript (ECMA)

With CADSE’s easy-to-learn

JavaScript, Tom further customized

guidelines to meet unique needs. He

created new objects, customize existing

objects, and programmed guidelines to

exhibit unique behaviors or to simulate

special processes.

If he doesn’t want to write script the

wizard copies his actions and produces

script for him

ADAPTIVITY PRINCIPLE

How to Implement Tom’s CADSE “Dream”

• Analyze Alternative Implementations? – Make vs. Buy(& integrate)– Usefulness, Scalability, & Usability

• Make Option: Case Study– Protégé = dozens of person-years of effort

(scalability & usability still issues!!)– One could spend all the budget and not be done!

The Buy (& integrate) Option

• Identify Best of Breed DSS Authoring Tools– Scalable, usable, generic, open-standards based

• Identify Next Release Change Requirements– Lobby effort needed, evolvable API concept

• Assure Workbench is an Integration Platform– Interface Definitions– Common Presentation Layer Issues– Fill In Missing Components

Goals of a Knowledge Management Workbench

• Top Down Support– Templates

– Reusables

• Sharing Existing Objects– Libraries

• Bottom Up Support– Free-form Creations

– Custom Designs

– Local Adapting

• Capture of New Objects– Learning

– Indexing

NOTE: Guideline Checking is an Extensive Topic for the Workshop

• Group Development, Version Tracking, & Administration

Power vs. Learning Tradeoff Curve

Power of Expression

LearnabilityShallow Workbench(visual, with wizards)

Powerful Workbench(programming lang’s)

Hybrid

How should a workbench support guideline creation efforts?

Pros Cons

One System Desirable,powerful

Never finished,very costly

Many LowLevel Tools

Flexible, righttool for job

Feature explosion,non-integrated?

WizardPrompting

Fast, effective,helpful

Guide to Guidelinesdoesn’t fully exist

ANSWER: A mix of approaches is required (HYBRID)

11Answer/Define

Information Gathering Questions

10View/Identify

Data Collection Needs

9Study/Draw

Temporal Dependency

Flows

3See/Fill In

Textual Annotations & Explanations

4See/Provide Encounter Materials

(Multimedia Links)

8See/Author

Pseudo Rules

6See/Fill out Procedure Triggers

7View/Create

Decision Tables

2Show/Draw Algorithm Diagram

(Adapt reusables)

5View/Add

Consultation Tables

Exchange Protocol Definition Models

Vocabulary Server

Semantics, Syntax, Structure

Models

Critics•Clarity•Completeness•Logic•Lexicon

CanonicalGuidelinesRepository

(CGR)

• Knowledge Mediators

• Info Agents• SNOMED RT• LOINC• CPT 4• NDC

• ARDEN syntax• HL7 RIM • GLIF & EGO

Parsers & Compilers• Syntactic/semantic checkup• Data & knowledge binding• Executable generation

VGR Administration

• Ontology• CBR /Search• Version

Test/Execution Environment

•Workability•Benckmarking/Quality Ass.•Integration (R2Do2,CDR, Careweb)

Guidelines DevelopmentCycle

Task Completion Wizards/Help

Interface Files

1Find/View Existing

Guidelines

Syntactic and Semantic Tools(Servers)

DeploymentTools

(HOLON)

GuidelinesDevelopmentTools (COPE)

TASK LAYER

TOOLS LAYER

Level 1Registration

12Make Guidelines

Computer Operational

Level 2Registration

Level 2

Level 3 Registration

Figure 1 - Overview of the Three Layers of the Knowledge Management: Tasks Ring, Tools Ring, and Repository Core

Outline






5. Clinical Pathways



• Summary

Starting a New Guideline• Pick a Type• Automatically Create a “Container”• Wizard Asks Author for Boilerplate*:

– Authors, Date, etc.– Target population, needs, & cost implications– Rate of change & Practice variation– Evidence (degree of vs. opinion)– Difficulty of development, maintenance, implementation

* - KPNW’s “CPG Policy & Procedures” Paper (7/98), p. 8

The Knowledge Model

• IoM’s 5 Guideline Types (others?)– screening & prevention, – diagnosis & pre-diagnosis mgt of patients, – indications for use of surgical procedures– approp. use of techs/tests for clinical care– guidelines for care of clinical conditions

The Knowledge Model (cont’d)

For each Generic Type of Guideline, Need to Model:

• Skeletal Plans/Chapters & Sections

• Sorted by Workflow Element (Person-type, Sequential Step, Timing Points, etc.)

• Locally Adaptable

Importance of a Layout/eForms Manager

• Adapt all eforms (container GUI, eligibility objects, reusable components)

• Use to Create Many Items– Progress Note Forms– Clinical Path Tables– Educational Materials– Other

• Use to Create Other Tools (e.g., PocketCards)

REQUIRED: Highly Scalable/Usable/Adaptable Forms Layout Manager

Design electronic forms in a Visual Forms Designer & fill them out using:•Web-based Visual Forms Builder •Visual Forms Application Builder •Visual Forms Webpage Plug-in•Workflow for eForm Artifact

Support for other Form Designer Tools and interchange standards

e.g., Visual Forms Designer can also read FormFlow, OmniForm and JetForm forms after they are converted via the Visual Converter.

Container Layout Manager/”Visual Forms Designer”

Uses Advanced Basic

Sample Visual Designer Screens

Examine some of the screens to assess role of clinician-authorsWww.mmacorp.comvfftw.htm

Make vs. Buy - Layout Manager

Decision CommentJetform orcomparable

Buy Assure open standardsare followed.Need to buildreusables library

Integrationw/Workbench

Make Seamless interfaceneededNon-uniformpresentation layer

Authoring Process:1. Protocol Eligibility

• At 1st Blush Eligibility Seems Straightforward– Inclusion Criteria– Exclusion Criteria (Mutable, Immutable)

• BUT, ….

There seem to be numerous “eligibility” type problems

• Standard inclusion/exclusion lists (“simple” branching)

• Level of care/risk stratification (multiple entry points, multi-workflows, more probabilistic?)

• Caremap progression checking (temporal issues)

• Health risk assessments (consumer-run pre-eligibility checks + lifestyle/self-care advice)

• Tools may differ somewhat for each

•Assessment During Rehab•Assessment After Discharge from Rehab

What kind of authoring will clinicians do?

• Decision tree? (e.g., Lumina/Spreadsheet)– has a well-grounded semantics– probability handled readily– fuzzy membership

• Visual Object Questionnaire “Programming” (e.g., Toolbook CBT)– may be the ECOLOGICAL artifact– convert to HTML/XML– what about script?

Case Study: “Uncomplicated Cystitis” Protocol Definition

• Eligibility Criteria– Female at or >18 and at or <65 years of age– History previous uncomplicated UTIs only– Classic Sx of <3 days: dysuria, freq. &/or nocturia

• Exclusion Criteria– Concurrent Coumadin Therapy– Pregnant (confirmed or suspicious) by history– Diabetes Mellitus, Renal insufficiency– UTI within last 6 months, or freq. post-coital UTI– Post-partum, less than 2 weeks …..

Decision Tree Buildup Process(I)Decision Analysis by TreeAge (http://www.treeage.com)

Decision Tree Buildup Process(II)

Decision Tree Buildup Process(III)

6 kinds of nodes

Eligibility Decision Tree

Advantages and Disadvantageswith Decision Tree (I)

• Advantages– Easy to create (reusable objects & wizards)– Table type input by user is efficient– Probability rollback procedure is a

GROUNDED SEMANTICS (extends GLIF)– Values can be queried from Patient Record – Can be implemented in certain ES shells

Advantages and Disadvantageswith Decision Tree (I)

• Disadvantages– Might not work with missing values– Might not interview at screen for missing values– Visual GUI is feature-rich (FEATURE

EXPLOSION) -- but less tedious than GLIF– How to parse Decision Tree output into GLIF (ES

shell rules are easier to parse)?– Often no visual developer GUI if in an ES shell

Same Case Study - Questionnaire Approach

• Asymetrix Multimedia Toolbook II – Used for Computer Based Training & Kiosks– Competitors - MacroMedia, HyperCard

• Output - HTML and Java

SCORE

Eligibility Tree - Make vs. Buy Alternatives

PROs CONsGLIFapproach

Merger of DT & QuestionnaireStandards-basedOther vendor usage

Expensive, time & costDuplicating other toolsWill never be as goodHard to maintain

DecisionTreePack

Usable GUIGrnded SemanticsXslates to Sprdsht

Can vendors use it?

QuestionnaireTool

Ecologic ArtifactXslates to HTML

What script runs it?What engine needed?

Authoring Process:2a. Wellness

• What’s involved here? Any more than:– Handout materials (education & DSS)– Show/Discuss Counseling pages– Provide Links to support & chat groups

• What examples?

• If the above process is correct, do authors need website wrapping tools? (eg W4F later)

Authoring Process: 2b. Prevention

• This is like the Government’s “Put Prevention Into Practice” Kits????– Lots of schedules (physicals, regular exams)– Immunization calls/timelines/reminding– Workflow and timelines

• call center

• case manager

• appointment clerk

• nurse

• doc

Visual Workflow - e.g., Filenet

While this may seem “intuitive”,

• HFHS Complains– Not intuitive– Too hard for their clinicians (WHY? Need to investigate)

• Possibility: No “model” of clinical world to map to– Outpatient clinic metaphor missing (=ecologic artifact)

• Work Performers = unmanaged spaghetti code

Alternative Timeline Management Metaphors

• Alternative 1 - Find a visual metaphor that cognitively fits each type of situation (DMI)

• Alternative 2 - Drive the interview for workflow info via a “knowledge model” of what needs to be collected (Agent I/F)

• A few ideas from R2Do2

The R2Do2 Agents from a Workflow View

Users, Patient,Doctor,Nurse, etc. Plan(ToDo Items)

UnPlan(Items)

Parser& BrokerRemoteRecords

Guideline Agents

(Short-lived,multi-thread)

ReminderAgents

(Long-lived,wake/sleep)

GuidelineKB Repository

Triggers,EventServer

Ongoing Alerts &Reminders

User AddsNotificationPreferences

Email, beeper, ITV, etc.

Age

7 years

DTaP (preferable) or DTP

DT

NoYes

Td

Contraindication to Pertusis

Case Study: The R2Do2 Immunization Agents’ Knowledge Base -- DTP Vaccine

Figure 3.1 Dependency Diagram of Logic for DTP Immunization

Contrain-dication

Vacc.Type

Recom.Schedule

Interval_from_previous_dose

age (in year, in month, and in week)

the number of prior doses

Vaccine_type of previous dose

Anaphylactic_

reaction

Encephalopathy

Table 3.2 Schedule for DTP vaccinationNumber of

prior vaccination0 1 2 3 4 Adult Booster

at birth First - over Dose

1 month Await

2 months First Second over Dose

3 months Await

4 months Second Third - over Dose

5 months Await

6 months Third First-Bo Over

Await Dose

15 months First -

Booster Second-

18 months Booster

First- Await

4 years First Second Third Booster Second

5 years available available Available available Booster

6 years

7 years Adult- Booster

Await Every 10 years

Metaphor that usesgraphics, 2-D seq,table format, andcolor to capturetemporalities(still need to addpersonnel resp.)

Table 3.1 Routine DTP vaccination Schedule (Source: MMWRa)

Dose Age Customary age / interval

Primary 1 2 months Age > 6 weeksPrimary 2 4 months 4-8 weeks after first dosePrimary 3 6 months 4-8 weeks after second doseFirst Booster 15-18 months 6-12 months after third doseSecond Booster 4-6 years No necessary if first booster is

administered after fourth birthday.Additional booster Every 10 years after last dose

Preceding Picture Conveys Much More Info Than a Table Format

Figure 3.4 Determine the recommended schedule of DTP vaccination.

if interval_of_separation >= minimum_wait thendue := “ now. ”;conclude true;

elsedue_date := time of (minimum_wait - interval_of_separation );due := “ due on ” || due_date || “ .”;conclude true;

endif;

prior_no sequence_no minimum_wait interval_of_separation

0 “first” 6 weeks age_in_week1 “second” 4 weeks interval_in_week2 “third” 4 weeks interval_in_week3 “first booster” 6 months interval_in_month4 “second booster” 4 years age_in_year

done “adult booster” 10 years interval_in_year

The semantics are of a temporal logic

Consumer Immunization

Record

Receive

Birth date

SSN

Mother’sFirst Name

Mother’s Last

Name

Mother’s Maiden Name

Mother’s SSN

Father’s First Name

Father’s Last Name

Father’s SSN

Provider ID

Provider

Last Name

First Name

Practice Address

Practice City

Practice State

ZIP

Give SubID Counter

AdministrationSubIDCounter

Date and TimeStart

Administration

Date and TimeEnd

Administration AdministeredCode

AdministeredUnits

Administered

Amount

AdministeredDosage Form

AdministeredDosage Form

AdministrationNotes

Administeredat Location

AdministratingProvider ID

Administeredper

Time/UnitAdministered

Strength

SubstanceLot Number

AdministeredStrength Units

SubstanceExpiration

Date

SubstanceManufacturer

Name

SubstanceRefusal Reason

Indication

CompletionStatus

ActionCode

System EntryDate and Time

ObservationIdentifier

Units

ObservationValue

ObservationSubID

ReferencesRange

AbnormalFlags Nature of

Abnormal Test

ObservationMethod

Probability

Observ ResultStatus

Date LastObs Normal Values

User Defined Access Checks

ResponsibleObserver

Date and Time of the

Observation

ClinicalObservation

Person

A kind of

A kind of

Email address

HCO Group No

HCO User ID

Password

To Do Item

Remind

Description

Due Date

Lead unit

Interval

Interval unit

Next reminder

Repeat Count

Repeat unit

Lead time

Repeat interval

Reminder No Repeat until date

Subject

To Do IDState

Questionnaire

Possible Answer

Question

Questionnaire ID

Question ID

Fill out

Interface Object ID

Interface Object Type

Parent Question ID

Vaccine Manufacturer

Produce

Name

ID

Name

Vaccine Code

Description

Include

ALT 2 - Agent Uses A Model to Drive the Interview: CDC’s HL7 Spec for Immun.

Data

AlertRemind

IndicationContraindicationPrecaution

translated to

activate

refer

1..n

1..n

refered

referee

trigger

1..n

Evoke criteria

produce

originate

Action

UrgencyExecution methodDescriptor

execute_action

refer to

1..n

use

1..n

based on

1..n

contains

1..n

Response message

Message kindContainer

infer about

contains

Attribute driven logic

Prediagnostic stageDiagnostic stagePost Diagnostic

Temporal logic

Clinical Logic Segment

Requisite attributes set

Attributes set

retrieve(object attribute ,query conditions)

Operational representation

Representation schemaLast updatedContainer idRepresentation container

call

refers

Knowledge Element

TypeElement IDPriority

Maintenance Element

TitleFilenameVersionInstitutionAuthorSpecialistDateValidationElement ID

Library Element

PurposeExplanationKeywordsCitationsElement ID

Guideline

Guideline ID

Reference Information Model

PersonPatientPharmacy_Service_AdministrationIndividual_Healthcare_ProviderPatient_AllergiesClinical_Observation

Vocabulary/Lexicon

ICD10NDRICD9SNOMEDVocabulary/Lexicon elementVocabulary/LexiconVocabulary/Lexicon Server

Given a Generic Model, a Wizard Should Work for New

Vaccines

• Wizard will interview for all the temporal semantical elements, and whatever else…

• Can a generic model be extracted for all vaccines? (one works for all 7 current ones)

• How many other generic wizards must be created? - a research task (cost-plus, not FFP).

vCalendar The Electronic Calendaring and Scheduling Exchange Format

• Consortium Members:– IBM, Lucent, Lotus, Microsoft, Novell,

Netscape, Oracle, etc.

• Open grammar for personal organizers, professional schedulers, calendar software

• GUIs for this grammar have already sold millions (HIGH USABILITY)

R2Do2 Reminding & Calendar GUIhttp://hera.seas.gwu.edu/prevention/prevention.html

http://bartok.seas.upenn.edu:8080/

Kill

Recur ComputeNext (IAction)

Launcher Launch(PAS)

Monitor(k)

Sleep Monitor

Awaken Check

Set Alarm ComputeNext(ISleep)

Remind ConsumerProvider

AFTER(Now,ISleep), Monitor

OODBConsumer Data

- DONE(Pk)

User Input Collect Scheme

Update 2Do

FiArchive

DONE(Pk) &- NULL(ISeparation)

-HOLDS(Pk) or [DONE(Pk) &NULL(ISeparation)]

DONE(PNk)

Mk

IAction

ISleep

ISleep

UserPreferences

Markov Chain Oriented

Make vs. Buy - Maintenance Tools

PROs CONsBuy VisualWorkflow

Already existsHelp/Trainingprovided

HFHS ComplaintsOpen Workflow Stds are Limited

Build NewMetaphor

Cognitive fitHigh Usability Pot.

Research neededEffort & time

Add Cal.Grammarto GLIF

Enhances GLIFReuses R2Do2 code

Will GLIF acccept?Time & Cost

Authoring Process:3. Chronic (Ambul. Care) Guidelines

• What is the “Chronic Guideline” Generic Structure?

• Show an example Guideline Form – (eg guideline-form widget from Protégé)

• Walkthrough Jetform Demo for Layout Mgt

• Walkthrough all Parts/Layouts– Textual instructions sheets– Visual Annotated Algorithm (GLIF Pallette, Risk-Costs-Benefits Table)– Drill Down to Charting Forms– Other

Typical Annotated Algorithm & Table

•Visual flowchart usefule for elicitation •(wizard assisted, terminology-enabled, GLIF semantics)

•Visual flowchart vital for display to users•(save as a GIF or JPEG?)

•Annotations include extensive supplemental material•clinician advice pages•patient educational materials (tables, graphs, etc.)•etc.

•Parse flowchart into machine executable (behind scenes)

Role of GLIF

• GLIF is an object oriented proposal for the interchange of machine executable guidelines

• Guideline Interchange Format

Intended Benefits of GLIF

• Store guidelines in repositories and eliminate duplication of effort between institutions

• Ease of dissemination (and maintenance)

• Deployment as executables

Basic GLIF Model

Strengths of GLIF

• Focuses on Types of Steps (bottom LHS)– Well-Formed Semantics of Step Constructs

• Very Good for Sequential Algorithms

Example Annotated Algorithm in GLIF

Illustrative GLIF Pallet and Flowchart Drawing Tool

Poor Criterion Logic

R2Do2 Criterion Logic as a Model: Can Authors work with this?

http://hera.seas.gwu.edu/knowledge_factory/login.html

Limitations of GLIF (I)

• Drawbacks mentioned by the authors– Representation of medical concepts in absence

of a standard clinical vocabulary– Lack of a formal syntax for the representation

of conditional expressions– Need to express temporal information in

clinical guidelines– Recognize and handle uncertainties about

patient data

Limitations of GLIF (II)

• Class hierarchy is not clear– All the objects are inherited from Guideline

Model, but they share very few similarities

• Categorization of different steps– Easier to model simple algorithms than

annotations and complex guideline models– Many guidelines are loosely organized and lack

obvious causal effects between parts

Potential Improvements for GLIF(I)

• Revise the class structure and add more relationship representation between objects– basically GLIF can only model steps and their

links (fares poorly on many other relations)– guideline must offer a containers to hold all the

relevant information, definitions, etc. (EGO)

Potential Improvements for GLIF(II)

• Increase data structures– Table Structures (Clinical Paths, C/B Risk)– Equations, Continuous Graphs– Graphics

But all of this is presumes a NARROW INTERPRETATION

of Guideline SemanticsWhat is a BROADER

INTERPRETATION?????

Well for one thing, look at what CANNOT be supported by GLIF

semantics

Weaknesses, What GLIF Omits

CADSE requires research on high level clusters of GLIF semantics

Grounded Needs Grnding

Guideline 5 Types of GL

Skeletal Os Dec.Tree,Markov Chain

Ecol.Artifacts:(Clin.Path, Progr.Note, etc)

Steps &Stmts

Branch, Crit.,Cond,

Time,

Logic Terms, vCal,Booleans

Criterion logic,

What is the “Chronic Disease Guideline” Generic Structure?

• SOAP(IE) too individualized

• E&M Coding too high level

• Allan Khoury’s hospital “progress note”– diseases, problems, complaint– HPI– Review of Systems– Physical Exam– Impression/Plan– Orders

• Other? (eg., the above ignores workflow)

Design GUI for each step of interviewing process. (Customizable)

Insert Charting Objects (Using an eForm Layout Manager)

PATIENT CHART TEMPLATE

Patient Name _______________________________ Acct. Number _____________________ Date _____ /_____ /_____

PATIENT HISTORY

ALLERGIES ______________________________________________________________________________________

CC _____________________________________________________________________________________________

HPI _____________________________________________________________________________________________

________________________________________________________________________________________________

(location/duration/quality/context/severity/timing/modifying factors/associated signs and symptoms) or (three chronic orinactive conditions)

PFSH: Past__________________________________________________________________________________________

Family______________________________________________________________________________________________

Social______________________________________________________________________________________________

REVIEW OF SYSTEMS: (+, -, na ) NOTES

_____Const _____Musc/skel _____Eyes _____Skin _____ENT _____Neuro _____Cardio _____Psych _____Resp _____Endocrine _____GI _____Hem/Lymph _____GU _____Allergy/Immuno

_____ See additional notes

COMPLEXITY OF MEDICAL DECISION MAKING

CODABLE 1 _____________________________________ 2 _________________________________

Diagnoses/

symptoms: 3 _____________________________________ 4 ______________________________________

RULE OUT

Diagnosis 1 _____________________________________ 2______________________________________

Management Plan/Orders/Medications:

Tests ordered (reference codeable DX for each), performed, reviewed and old records reviewed:

Instructions/Counseling/Coordination of Care:

Time: in _________ out _________ with pt _________ Physician’s Signature____________________________________

Assessment of pain intensity and character:(QUESTIONNAIRE)

1.Onset and temporal pattern

When did your pain start? How often does it occur? Has its intensity changed?

2.Location

Where is your pain? Is there more than one site?

3.Description

What does your pain feel like?

What words would you use to describe your pain?

4.Intensity

(see next page)

5.Aggravating and relieving factors

What makes your pain better? What makes your pain worse?

6.Previous treatment

What types of treatments have you tried to relieve your pain?

Were they and are they effective?

7.Effect

How does the pain affect physical and social function?

InteractiveGraphic

PATIENT INSTRUCTION

Make vs. Buy - Annotated Algorithm

PROs CONsGLIF as-is Free

De facto standardMust build an EGO toimplement

GLIF adds(cond.logic& Tables)

Added power &usefulnessCan have good GUI

Intermed acceptability?

ProgressNotes

Highly flexible(use Layout tool)

We must buildAssure Open-std - XML

Authoring Process:4. Acute Care Guidelines

• Triage in the ER

• Indications for Use of Surgical Procedures

• Appropriate Use of Specific Technologies and Tests as Part of Clinical Care

• Outcomes capture and analysis

Website Wrapper Tools

Also for Extraction of Text/Data

Building a wrapper for theNational Guideline Clearinghouse

• We present how to use the our Wrapper Factory (W4F) to write a wrapper for the National Guideline Clearinghouse.

• Our wrappers consist of – a retrieval layer (to fetch the remote Web document)– an extraction layer to extract information from the HTML source– a mapping layer to export the data to higher-level applications

• As a concrete illustration of extraction, we want to build an XML document that briefly describes the guideline.

Retrieval Rules

Extraction Rules

Mapping Rules

NSL

NSL

NSL

INPUT HTML treeHTML document

title

date

availability

Parser

Web

XML documen

t

Step 1: grab a guideline sample• Identify a Web page that describes a guideline

• Guidelines should all have (more or less) the same structure

Step 2: Feed the guidelineinto the extraction wizard

The extraction wizard• The extraction wizard returns the Web document with some invisible

annotations.

• The display is identical to the original document.

• Text areas get highlighted when the user puts the mouse over them

– it helps the user identify information boundaries

– it tells the user how to reach this piece of information using the hierarchy of the document

Step 3: Define some extraction rules• Using the information provided by the extraction wizard, one can

now write the wrapper.

EXTRACTION_RULES{ all = html ( ->a[i]->pcdata[2].txt # ->a[j]->pcdata[2].txt # ->a[k]->pcdata[2].txt # ->p[l] ( .txt # ->a[1].getAttr(href) # ->p[m].txt ) ) where html->a[i].getAttr(name) = "'title'" and html->a[j].getAttr(name) = "'references'" and html->a[k].getAttr(name) = "'release_date'" and html->p[l].b[0].txt = "GUIDELINE AVAILABILITY:" and html->p[l]->p[m].txt =~ "Print copies";

}

The output XML document and its DTD<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE W4F_DOC [

<!ELEMENT W4F_DOC (GUIDELINE)>

<!ELEMENT GUIDELINE (REFERENCES,RELEASE_DATE,AVAILABILITY)>

<!ATTLIST GUIDELINE TITLE CDATA #IMPLIED>

<!ELEMENT REFERENCES (#PCDATA)>

<!ELEMENT RELEASE_DATE (#PCDATA)>

<!ELEMENT AVAILABILITY (TEXT,URL,COPY)>

<!ELEMENT TEXT (#PCDATA)>

<!ELEMENT URL (#PCDATA)>

<!ELEMENT COPY (#PCDATA)>

]><W4F_DOC> <GUIDELINE TITLE="Prevention of plague: recommendations of the Advisory Committee on Immunization Practices (ACIP)."> <REFERENCES>MMWR Morb Mortal Wkly Rep 1996 Dec 13;45(RR-14):1-15 [37 references]</REFERENCES> <RELEASE_DATE>1996 Dec 13</RELEASE_DATE> <AVAILABILITY> <TEXT>GUIDELINE AVAILABILITY:Electronic copies: Available from the Centers for Disease Control and Prevention (CDC).</TEXT> <URL>http://…..</URL> <COPY>Print copies: Available from the Centers for Disease Control and Prevention, MMWR, Atlanta, GA 30333. Additional copies can be purchased from the Superintendent of Documents, U.S. Government Printing Office, Washington, DC 20402-9325; (202) 783-3238.</COPY> </AVAILABILITY> </GUIDELINE></W4F_DOC>

Work to be completed on Penn’s W4F

• Classic Power vs. Usability Tradeoff– Working and Scalable (wrapper is for programs)– No GUI yet available (not usable by clinicians)

• Other Research Issues ...

Authoring Process:5. Clinical Pathways

• Reusable Tables and Other Objects

• Several Options to Alter Objects:– Wizard interview to change them– Control Preference Screens to change them– Direct manipulation

• Objects self-parse into class hierarchies, XML sheets, precedence relations, & workflows

Typical TableBuilder Screenshot

Day 1Admit

Day 2Surgery

Day 3ICU/Rm

Day 4Xfer

ConsultsTestsActivityTreatmentsMedsDietTeaching

Authoring Process:6. Education/DSS material

Educational Materials Generation

• Education Materials Text/Table Construction– Clinician DSS Pages (eg, evidence tables)– Patient Education Handouts

• Useful Toolset– Type and format– Cut & Paste from other sources– Webpage authoring (e.g., Netscape Composer)

• Also Publishing the Concept Map/Website Semantic Net (RDF) is helpful for maintenance

Outline









• Summary

Maintenance/Admin Requirements

• Guideline Keeper: KPNW P&P pages 13, 14, & 23– consulting expert on that guideline– exception & outcome tracking– literature surveying/detect evidence shifts– recommends guideline updating

• Want tools to help him/her

Maintenance/Admin Tools• General literature tracking tools

• User/Exception/Outcome reports and fixes tracking

• Administrative Tools– set access privileges/maintain security keys

– track usage/adapting of guideline objects/versions

– manage library (checkin, checkout) of diverse containers, components, reusable objects, BLOBs (unlike Ontyx’s homog. term graph)

– inter- and intra-guideline error checking

Traditional Maintenance Paradox

• Project Leaders and Organizations pressure project staff to complete software tasks with minimal budget and delay

• This savings in the development cycle is the very thing that leads to unmaintainable code downstream

• Maintenance is 50-90% of most SW project budgets

What are the Maintenance Needs and How can We Predict Them?

• Examine Case Studies

• Study literature

• Find Principles/Methods (e.g., SEI Capability Maturity Model)

Case Study: DEC’s XCON Expert System

• XCON = VAX config expert system (1983-present)

• One of the world’s 1st expert systems (largest)

• 12,000 rules, 69 locations, released 3x/year

• Vastly successful (configuration quality and sales)

• All production operations=70 mission-critical ESs

• 10,000 users, 250 staff, $200 million/year savings

• kept DEC afloat for many years

XCON Maintenance Nightmares

• Domain misunderstood - VAX change rate overlooked, clusters unhandlable, rules not reusable for other than VAX 11/780s

• Poor Make-Buy Decision - OPS Shell was inexpensive. They allowed it to drive design

• Employee turnover -- no one knows entire KB, can’t remove another’s rule easily, little documentation (as always)

• Irony - unreadable, high level KB

XCON’s Maintenance Solution: Basic Reading & ‘Riting (2Rs)

CLARITY ENFORCING EDITORMaster Structural TemplateOne Partition, One Problem

Domain-Specific Problem Solving MethodsOne Function Per Rule

Legal Syntax for Each Rule

OPS5 Knowledge BaseAuthors

Maintenance Case Study 2: The COPE Kn. Management System

• Automatically help Army to generate, critique, index, and maintain ~600 new guideline type documents/year

• Use of DDN (classified Internet, pre-Web)• Started 1989= just ahead of C++ and Windows• Fielded 1991 (1992 IAAI awardee)• Major code in C++ & Windows Precursors• Code Maintenance Issue: How to convert seamlessly

to C++ and Windows

How Did COPE Handle Maintenance?

• 5,000 Chunk “Static KB” = basic guidelines on document critiquing & document gen. Wizards– Published index/concept map (automated)– On-line updating tutorials– On-line updating editors/critics

• Direct manipulation GUI at XCON’s R2 level

– Color and textual cuing (critiquing, feedback)

How Did COPE Handle Maintenance (continued)?

• Dynamic KB of 600 new documents/year parsed and stored as OO annotated decision algorithms

• World Model concept map & editor (like CMT server and tools)

• Automated indexing (down to reusable sentence structures) via Case Based Reasoning (CBR) algorithm

• Password protected signature chain for KB version updating

• Use of CBR to push reusables to users

Make vs. Buy - Maintenance Tools

Decision CommentOntyx VersionController

Buy Will it work on heterog.Object library?

XCON-R2Type Editor

Make/Buy

This should be CADSEitself

Wizard Editor Buy?

CBR Tool Buy May have to embellishand integrate

Guideline Debug & Check

Outline









• Summary

What we covered today• Workbench challenges

– usefulness, scalability, usability– power vs. learnability– make vs. buy

• Dream CADSE scenario

• Workbench Requirements:– What are you trying to put in front of the clinician-author – What should they put in front of clinician-practitioners to make

right thing easiest thing

What we covered today (continued)

• Common, Ecological Artifacts Need to be Authored for a Range of Guideline Types (chronic, acute, prevention, wellness, educ, etc.)– eForms, GUIs

– workflows and calendar/temporal metaphors

– algorithms & their annotations

– eligibility/assessment trees & questionnaires

– table structures (ClinPath, Risk, Dosage, etc.)

– educational materials

What we covered today (continued)

• 15 Tools of the workbench– Breadth -- which ones to include– Depth -- how to ground &

implement them

Make vs. Buy - Summary

Parts to Buy Parts to MakeWorkbench JDK, Oracle8 Envmt & I/FsType Contaner JDK eForm,ClassesLayout Mgr JetForm? reusablesElig.Tree Tool TBD? manyWorkflow OpenWkfl Std

Cal.GrammarMetaphors

Annotated Alg GLIF GLIF AddsWrapper Tool K2, Ariadne GUI, otherClinPath Tool Wizard Genrtr GUI/ReusblesMaintenance CBR, Ontyx World model

Hardest Research Challenges Identified Today

• PRAGMATICS: “Discovering” ecological artifacts– How to best help authors create guidelines materials– Clinician epistemology (don’t know what they know)– In lieu of 1st princ, at regular intervals need: interviewing time,

situational experiments, feedback discussions

• SEMANTICS: Enhancing GLIF “Layers”– Guidelines, Skeletal Parts, Steps, Logic

• SYNTAX: Creating OO implementations (tools) with direct manipulation GUIs and wizards– useful, scalable, usable

What Did we Skip Today

• Integrating the Term Picker*• Groupware Editor*• Guideline Checking (lengthy topic)*• Guideline Executables (EGO Class Library)*• Order Feedback Rules -- Author, Make/Buy*• Knowledge Mediation Layer (lengthy topic)*• Update on Make vs Buy Investigations*• Other?

* - Topics to be covered at length in the 2-day Phila. workshop