"Improving decision-making and workflow in patient care: a review of PROforma technology and its current evidence base"

Cancer Research UK* has been developing the PROforma language for modelling clinical processes and associated decision support and workflow technologies for about ten years. During this period the foundations of the language have been formally established, clinical applications and development tools have become increasingly flexible, and a range of applications have been built and clinically tested. Applications to date range from suport fpr prescribing and referral decisions in general practice to management of cancer and HIV+ patients. This talk will summarise the concepts underlying PROforma and review seven empirical trials which have provided quantitative data. These results strongly suggest that appropriate technologies can yield major benefits in consistency, quality and safety of patient care, together with improved resource management and good clinical acceptability.

• More information about PROforma and other related technologies can be found at www.openclinical.org, and PROforma in particular at www.openclinical.org/gmm_proforma.html

• *previously Imperial Cancer Research Fund

Improving decision-making and workflow in patient care: a review of PROforma technology and its

current evidence base"

BCS 2005

“Medicine is a humanly impossible task”

• Up to 97,000 unnecessary deaths p.a. in the US are due to medical error. Total national costs … between $17B and $29B

Institute of Medicine, To Err is Human 1999

• In NHS hospitals “overall rate of preventable adverse events of 11.7%. [A third of which] led to …disability or death, … others are frequent, minor events … but together have massive economic consequences”

Vincent et al, BMJ 2001

… and in oncology …

• “Perhaps 16,000 lives could be saved if all current knowledge of cancer were properly applied”

ICRF Vision for Cancer, 1995

• “There have been undoubted improvements in service delivery but there is still a sense that progress has been patchy and that much has yet to be achieved.”

National Service Framework Assessment of NHS Cancer Care 2001.

Promoting best practice

… but

  Busy clinicians have little time to read

  Even if there is time, memories are unreliable, working pressures acute

  Conventional guidelines address general principles of care, not the needs of individual patients

Support at the point of care

R Steele et al, Proc. AI in Medicine Europe, 2003

-+

The PROforma method

What is PROforma?

Before musical notation, every singer had to memorize the entire repertoire. Those singers then went on to teach the next generation. Small errors in memory or differences of taste caused the chants to change over the years and no two singers would learn a chant precisely the same way. Notation made it possible to record a chant in a definitive form for easier and more reliable communication.

Guido d'ArezzoBenedictine monk, musical theorist and teacher.

The PROforma language is based on an ontology of

general tasks

Generic task

“Keystone”

Enquiries PlansDecisionsActions

The Tallis toolsetComposing and publishing clinical guidelines, protocols and pathways

plan :: 'plan2' ; caption ::"Chemotherapy"; description ::"Care pathway for chemotherapy"; precondition :: result_of( decision2) = Chemotherapy ; component :: 'action2' ; number_of_cycles ::1; component :: 'plan5' ; schedule_constraint :: completed('action2') ; number_of_cycles :: 3 ; component :: 'action4' ; schedule_constraint :: completed('plan5') ; number_of_cycles ::1;end plan.

decision :: 'decision1' ; caption ::"Diagnosis?"; description ::"Differential diagnosis between cancer and peptic ulcer."; candidate :: 'peptic_ulcer' ; argument :: for, ( biopsy = negative ) argument :: for, ( pain_time = delayed ) argument :: for, ( age = young or age = adult ) argument :: for, ( pain_site = epigastric ) recommendation :: Netsupport( decision1, peptic_ulcer ) >= 1 ; candidate :: 'cancer' ; argument :: for, ( biopsy = positive ) argument :: for, ( pain_site = epigastric ) argument :: for, ( age = elderly ) argument :: for, ( smoker = yes ) argument :: for, ( pain_time = immediate ) recommendation :: Netsupport( decision1, cancer ) >= 1 ;end decision.

action :: 'action5' ; caption ::"Medication"; description ::"This (dummy) action is carried out if the diagnosis is peptic ulcer"; procedure ::'Medication';end action.

action :: 'action1' ; caption ::"Refer to surgeon"; procedure ::'Refer to surgeon';end action.

enquiry :: 'enquiry2' ;

caption ::"'Measure wbc'"; description ::"Chemotherapy: record white blood count"; source :: 'wbc' ;end enquiry.

Repertoire: knowledge bases of standard reusable components

Protocols, guidelines, care pathways

Formalised in PROforma

Tested in silico (“on the bench”)

Routine use

Feedback into research and policy

Trials

Cost-benefitanalysis

Cost-benefitanalysis

The “figure of eight” model

Applications and evidence

Prescribing in general practice (CAPSULE)Walton et al British Medical Journal 1997

Automated image interpretation Paul Taylor, Andrew Todd-Pokropek, Medical Image Analysis (2000)

BrCa pathway

RAGs: Risk Assessment in Genetics Andrew Coulson, Jon Emery, David GlasspoolBMJ 1999; 2000; Meth. Inf. Med 2001

ERA: cutting waiting times Jon Bury, Michael Humber

BrCa pathway www.infermed.com/era

ALL Dose adjustment studyJ Bury, C Hurt, A Roy, L Cheesman, M Bradburn, S Cross, J Fox, V Saha (submitted)

Objectives: – To assess the clinical value of a decision support system designed to assist with dosage adjustments during maintenance therapy for childhood Acute Lymphoblastic Leukaemia.

Decision model: – one PROforma decision task, 8 options. Each has between 1 and 5 criteria associated with it, each referring to different clinical situations, expressed in terms of 5 parameters.

Materials and methods– Balanced-block crossover experiment, in which 36 clinicians with varying degrees of experience were asked to decide on oral chemotherapy dosages for 8 simulated cases: 4 using decision support and 4 without. – Outcome measures were number of protocol consistent dosage decisions; time to manage each case; accuracy of dosage calculations and clinicians' opinions about the value and usability of the system.

Dose adjustment in chemotherapywith CRUK Paediatric Oncology Group, London Hospital

Bury, Hurt et al, Proc. American Medical Informatics Association, 2002Hurt et al, Proc. AI in Medicine Europe, 2003 Bury et al, British Journal of Haematology (in submission)

ALL Dose adjustment study – resultsJ Bury, C Hurt, A Roy, L Cheesman, M Bradburn, S Cross, J Fox, V Saha (submitted)

Measure Without DSS

With DSS

Number of erroneous prescriptions

54/144 0/144 p<0.001

Number of times users deliberately overrode the protocol

6/144 7/144

Time taken to reach a Novicesdecision for each case Experts

156.5s110.8s

125.4s133.6s

p = 0.02P = 0.02

35/36 subjects said they would be likely to use the system if it were available

Measure Without DSS

With DSS

Number of erroneous prescriptions

54/144 0/144 p<0.001

Number of times users deliberately overrode the protocol

6/144 7/144

Time taken to reach a Novicesdecision for each case Experts

156.5s110.8s

125.4s133.6s

p = 0.02P = 0.02

Triple Assessment studyC Hurt, V Patkar, R Steele, T Rose, M Williams, J Fox (report in preparation)

Objective: – To evaluate the potential effect of PROforma decision support on clinical decision making with respect to national guidelines for Triple Assessment.

Decision model– Pathway included 4 decisions (familial risk, type of imaging, type of biopsy, and management).

Materials and methods:– 15 hypothetical paper cases covering range of clinical scenarios developed by an expert panel of five judges (2 breast surgeons, 1 breast pathologist, 1 radiologist and 1 geneticist) and optimal management for each case was agreed by consensus.– 24 doctors asked to manage 5 cases with and 5 without computer support. A balanced block design used to allocate cases.

Triple assessment of breast cancer

Hurt C, Patkar V, Steele R, Rose T, Fox J (in preparation)Steele R, Fox J Proceedings of European conference on AI in Medicine, 2003

24 participants (17 consultants, 5 specialist registrars, 1 nurse 24 participants (17 consultants, 5 specialist registrars, 1 nurse practitioner)practitioner)

Average number of years in speciality = Average number of years in speciality = 9.39.3 (range 1 - 23) (range 1 - 23)

Deviations / errors Without DSS With DSS Total decisions

All deviations 60 (50%) 16 (13%)

120In each

armCritical errors 10 (8.3%) 1 (0.8%)

Triple Assessment Study – ResultsC Hurt, V Patkar, R Steele, T Rose, M Williams, J Fox (report in preparation)

Would patient care improve with TA decision support?Would patient care improve with TA decision support?

In favour 16In favour 16 Undecided 1Undecided 1 Against 7Against 7

Would they personally wish to work with TA decision support?Would they personally wish to work with TA decision support?

In favour 12In favour 12 Undecided 3Undecided 3 Against 9Against 9

Deviations / errors Without DSS With DSS Total decisions

All deviations 60 (50%) 16 (13%)

120In each

armCritical errors 10 (8.3%) 1 (0.8%)

65 Decisions

• If compliance with best practice is 99% then 50% of women will get “perfect care” (0.9965)

• If compliance is 95% then 3% of women will get perfect care (0.9565)

• Our results from several studies suggest actual deviations from quality standards are between 10 and 30%

Other projects (InferMed 2003-4)

• Hoffman-La Roche – Retrogram®– Trans-national study (ORAMA) on Acute Renal Anaemia

• Brown University Pain Management, Long term care of the elderly.

• Mater Misericordiae Dublin – 2 electronic guidelines

• New Zealand Ministry of Health – Diabetes Management in General Practice

• Eclipsys Inc. – Integration with HIS• Pfizer – 3 Post-Op pain management guidelines

(PROSPECT)

Management of HIV+ patientsTural et al, AIDS, 2002, 16, 209-218.Retrogram® for Roche www.retrogram.org

Arezzo (3)

A standard format for guidelines and care pathways?

Research/centres ofexcellence

Specialistservices General

hospitals

Primary care

Home and self care

More informationwww.openclinical.org