EU Projects and The Learning Health System.

Brendan Delaney Chair in Medical Informatics and Decision MakingDirector Centre for Patient Safety and Service QualityImperial College London

What is a ‘Learning Health System’?

‘A learning healthcare system is [one that] is designed to generate and apply the best evidence for the collaborative healthcare choices of each patient and provider; to drive the process of discovery as a natural outgrowth of patient care; and to ensure innovation, quality, safety, and value in health care.’ Institute of Medicine. 2007

www.transformproject.eu

€7.5M European Commission March 2010-Nov 2015Funded under the Patient Safety Work Program of FP7

• To support clinical diagnosis• To support clinical trials• As part of a ‘Rapid Learning Healthcare System’

3

21 Partners from 10 Member States

Aims of TRANSFoRm

To develop methods, models, services, validated architectures and demonstrations to support THE LEARNING HEALTH SYSTEM:• Epidemiological research using primary care records,

including genotype-phenotype studies and other record linkages

• Research workflow embedded in the EHR• Decision support for diagnosis

TRANSFoRm clinical requirements (2010-15)

Part of a Learning Health SystemPrevalent and incident case identification from live EHR

systems in primary careReal time alerting via EHRPre-population of CRFs displayed “within” the EHRData capture in EHR (eSource)PROMS data in EHR for Safety monitoringData provenance – towards compliance with 21 CFR

Part 11 and European regulationFull evaluation in 5 EU member states and real world

RCT

TRANSFoRm technical requirements

To use ontologies to maintain models of meaning for the LHS (CRIM, CDIM, Provenance)

To use CDISC foundational standards ODM, SDMTo enable connection to multiple country, multiple

language, multiple vendor systems with MINIMAL vendor input – No ‘hackathons’

Vendor requirements:• Standard Terminology used in EHR (LexEVS)• Sample EHR data set• Represent local database metadata as a model (DSM) and

map to TRANSFoRm Clinical Data Integration Model (ontology)

• API and a demo installation for testing

Limitations of existing CDISC standards for clinical trials

Operational Data Model operates as a holder and can contain a wide variety of content.

Therefore clinical meaning and context can vary between clinical and research domains in unpredictable ways.

CDASH offers a means of assembling libraries of well-defined forms, but does not implicitly translate to a clinical setting

8

Approach to Meaning

TRANSFoRm uses common data models expressed as ONTOLOGIES to maintain meaning across the Learning Health System

Archetypes are used to link ‘meaning’ as a reference to the ontologies (Research and clinical meaning) at individual data element level

9

A software platform for the Learning Health System is built using tools to author, exchange and act upon these basic information constructs. It serves all three use cases.

10

Four points of eCRF/EHR integration

1. Extracting study data from the EHR to facilitate data entry

2. Asking physician to complete the study data that are not present in the eHR

3. Presenting all study data to the physician for approval before sending to the research databasein order to increase control and data quality

4. Adding care-relevant study data to the EHR, so that the physician needs to fill out the information only once

High level system architecture

ODM in TRANSFoRm

SystmOne pop up

SystmOne Eligibility

SystmOne Agreement

User Interface Extension to ODM

ODM extended with GUI elementsWhen an ODM is created, the QuestionType attribute is added to every

ItemDef object in ODMPlatform-agnostic, rendered appropriately for each device

18

A Learning Health System for Diagnosis

The power of early hypothesesHypotheses formulated quickly and with little information.1

Disproportionate influence of early hypotheses on subsequent judgements and decisions.2-4

Strong association between physicians’ initial diagnostic impressions and their final diagnoses (and management) in common presentations that could indicate cancer.5

Baron (2000);1 Asch (1946);2 Crano (1977);3 Forgas (2011);4 Kostopoulou, Sirota et al MDM 2016 5

The principle of early diagnostic support

• Suggest possible diagnoses early in the consultation, before physicians start gathering information to test their own hypotheses.

• Diagnostic suggestions based on patient’s age, sex, risk factors and reason for encounter (RfE).

• Principle tested in 2 RCTs: UK1 and Greece2 (D2.1)

• Online, interactive, simulated consultations with 9 clinical scenarios

Trigger DSS in Vision EHR

Early support

Coded data entry

Information transferred to Vision EHR

DSS prototype evaluation in a high-fidelity simulation

• 34 GPs – users of Vision EHR system• 12 standardised patients (actors)• 2 sessions (within-participant design)

» 1st session: (baseline performance) Vision EHR – no DSS.

» 2nd session: Vision EHR with integrated DSS.

Results

MEASUREBaseline

Mean [95% CIs]DSS

Mean [95% CIs]Odds ratios or

stand. coefficients[95% CIs]

Diagnostic accuracy

0.50 [0.42 to 0.57]

0.58 [0.52 to 0.65]

OR 1.41 [1.13 to 1.77],

P<0.01

Appropriate management

0.59 [0.52 to 0.66]

0.66 [0.52 to 0.65]

OR 1.34 [1.01 to 1.78],

P<0.05

Diagnostic certainty (0-10 VAS)

7.61 [7.28 to 7.94]

8.01 [7.79 to 8.23]

Beta 0.39 [0.12 to 0.67],

P<0.01

Data items coded into the EHR

1.64 [1.10 to 2.18]

12.35 [10.82 to 13.87]

Beta 10.71 [9.06 to 12.35],

P<0.01

95% CIs are adjusted for clustering on GP

Results

MEASUREBaseline

Mean [95% CIs]DSS

Mean [95% CIs]Standardised coefficients

[95% CIs]

Investigations ordered

2.51 [1.92 to 3.09]

2.83 [2.33 to 3.33]

Beta 0.33 [-0.54 to 1.20]

Consultation length (mins)

13.73 [12.61 to 14.85]

14.42 [13.05 to 15.79]

Beta 0.69 [-0.28 to 1.67]

Patient satisfaction(5-point Likert)

3.26 [3.07 to 3.45]

3.26 [3.10 to 3.42]

Beta 0.001 [-0.18 to 0.18]

Web service for DSS

3

Reusable ontology model of evidence

4

Data Mining: Steps

Web tool (clinical evidences)

Web tool (RuleViewer)

KNIME tool

ImportXML

CSV

Encounter data

Encounter data

TransHISCalculate

quality measures

Derive association

rules

1 2

3

4

5

Clinical review

Filter based on high

quality rules

Evidence transfer to ontology

5

Conclusions • Improvement in diagnoses and decisions

• Without more investigations ordered/time taken

• A lot more data coded into the EHR • GPs recorded during the consultation (not at the end)• Opportunity to capture rich (and less biased) routine

data• A Learning Health System for diagnosis

Thank youBrendan Delaneywww.transformproject.euBrendan.delaney@imperial.ac.uk