Clinical Research Informatics in Pediatric Critical Care J. MICHAEL DEAN, MD, MBA KATHERINE SWARD,...

Clinical Research Informatics in Pediatric Critical Care

J. MICHAEL DEAN, MD, MBA

KATHERINE SWARD, PHD, RN

Context

Critically ill and injured children typically receive care in the ED and/or the pediatric intensive care unit (PICU). A spectrum of heterogeneous conditions lead to need for “intensive care” : traumatic brain injury, other traumas, lung injury, sepsis, postoperative care

Epstein, D. & Brill, J.E. (2005). A history of pedicatric critical care medicine. Pediatric Research, 58, 987–996; doi:10.1203/01.PDR.0000182822.16263.3D

Unfortunately, much of the technology and many therapies in pediatric critical care have evolved without adequate study or have been adopted uncritically from adult, neonatal, or anesthetic practice…

NIH/NICHD has a new branch (Pediatric Trauma and Critical Illness), recognizing the need for research in this environment.

Rigorous use of appropriate scientific methodology, deployed across a network structure, achieves the numbers of patients required to provide answers...

http://www.nichd.nih.gov/research/supported/Pages/cpccrn.aspx

http://www.pecarn.org/http://cpccrn.org/

Context Clinical Research Informatics involves the use of informatics tools and methods in the discovery and management of new knowledge relating to health and disease.

It includes management of information related to clinical trials and … secondary research use of clinical data.

http://www.amia.org/applications-informatics/clinical-research-informatics

Context Intensive Care Informatics – a relatively new working group in AMIA

Data Coordinating Center

U of Utah Division of Pediatric Critical Care encompasses (but is not limited to): clinical services, Intermountain Injury Control Research Center, and a data coordinating center that supports a large number of research networks and clinical studies.

Informatics tools and methods are threaded throughout DCC activities study conception & protocol development LaTex, GitHub data models data collection standard CRFs, TrialDB OpenClinica RedCap CheckBox custom software development data quality QueryManager data analysis and publication network communications and logistics eRoom, teleconferencing regulatory/compliance monitoring; IT infrastructure (FISMA; HIPAA …)

http://medicine.utah.edu/pediatrics/critical_care/

Two examples PECARN: registry project (Dean) Pediatric emergency care applied research network.

CPCCRN: CDS tools (Sward) Collaborative pediatric critical care research network

PECARN•TBI prediction CDS

•Neuroimaging decision rule

•Decision rule for intra-abdominal injuries

•Pediatric Emergency Care Quality *

http://www.pecarn.org/pecarnNetwork/documents/BrochureFall2013.pdf

PECARN Registry Protocol Objectives

1. Develop the registry by merging EHR data from participating EDs.

2. Use the registry to collect stakeholder-prioritized performance improvement measures

3. Report performance improvement measures to individual clinics and to sites and measure subsequent changes in quality performance.

PECARN Registry Protocol Study Procedures

Database ConstructionDeidentification ProceduresNatural Language Processing ProceduresDetermining Benchmarks for Report CardReport Card Feedback

PECARN Registry Protocol Database Construction

Identify potential sources of relevant data elements in the specific EHR at each site.Finalize the types of data elements that will be extracted.Extract data for one day of data at each clinical site.Transmit one day data to the DCC for de-identification.Establish de-identification procedure at each clinical site.Extract and de-identify one month data from calendar year (CY) 2012 at each site.Transmit one month de-identified data to DCC from each site.Finalize and test import procedures from one month extracts into Registry.Analyze frequencies of missing, out of range, or unexpected values for key data elements.

Extract, de-identify, and transmit entire CY 2012 from each site to the DCC.Create Registry with entire CY 2012 from all participating sites.

The processWHAT IT REALLY LOOKS LIKE

PECARN Registry ETL

PECARN Registry ETL

Report Card Development:Expert Panel

Panel met on 2/24/14 to review data for pre-selected Performance Measures

Develop “ideal” benchmarks

ABC calculated by stats, presented in summary & full documents◦ New methodology, difficult to understand◦ Works well for dichotomous; continuous causes confusion

Definitions of Performance Measures ever changing

Huge stats efforts

Report Card:Designing

Formatting the Report Card, Simplify

Finalize Performance Measure definitions (yes. Still changing…)

~17 Performance Measures, reported with 4-5 benchmarks, want visuals & simple reports

◦ What will we loose

Comparison with past data, visual, quick

Redesign data warehouse to improve performance

Examples of Report Card draft today

Differences between development & production

Automation (eventually)

Only data needed for report cards is kept ‘active’◦ 4 months of data kept in active database for trendlines◦ Older data is archived

Data is locked & not allowed to update after a cut-off◦ Includes grouped & manually derived data

Data becomes more static – no resubmissions past deadlines!

Next Steps Data collection

◦ “Real-time” monthly submission of 2014 data

Test the whole production cycle◦ What happens when we really do this? How does it look?

Report Card delivery◦ Automation is critical, several IT methods/approaches – will develop over

time◦ Start small – email a report card

Measuring provider improvement once they get a Report Card◦ Staggered starts

Collect Report Card feedback from providers◦ Implement into the Report Card – improve!

Future directions? Conversations about data for “this study” (Report Cards) vs “registry as a whole” (future uses, add-on projects)

Adding more sites? Roll-up?

Adding more data?

Using the data for clinical trials?

CDS tools

CDS tools• ICU is information-dense environment (information overload is likely)

• Many interventions in PICU were adopted from adult practice, neonatal practice, anesthesiology, or other areas – lack evidence from pediatric environment

• For many critical care conditions, the “intervention” is not single point in time, but is the cumulative effect of multiple decisions and actions across days

Critical Care Medicine, 2008

Even for a relatively simpleprotocol like glucose/insulin

A single “decision” in reality requires multiple steps thatare conducted in sequence.

Challenge – production rulessystems like Drools are designedto run every rule at the same time.

Replicating eProtocol-insulinallowed us to validate the Java/Drools approach – could wegenerate the same recommendationsin both versions of the software.

A subsequent project (Hypertonic Saline) allowed us to examine issues related totiming – labs not synchronous wclinical data entry

Each node is a set of

rules

Next project : ventilator management for ARDS/ALI

ARDS in children•Estimates of ARDS in children range from 1.4 – 2.8% of all PICU admissions•Estimated: 3-4 ARDS cases per year per 100,000 population < 15 years of age•The one prospective US publication suggested a rate triple this at 9.5 per year per 100,000 admissions

•Therefore, likely there are 1800 – 5700 cases per year in US pediatric population < 15 years of age (2011 US Census = 60*106)

•Hence, ARDS remains a significant Public Health issue as overall there is a significant (~18%) rate of mortality

•ALI is less severe form…even more common?

Adult vs Pediatric ICU Developmental differences and differences in clinical practices may contribute to the selective application of evidence derived in other settings.

Ventilator modes such as high-frequency oscillatory ventilation (HFOV) are more common in the pediatric setting than in adults, for example; while invasive arterial monitoring is increasingly less common in the PICU.

Khemani, R & Newth, CJL (2010). The design of future pediatric mechanical ventilation trials for acute lung injury. Am J Respir Crit Care Med 182, 1465–1474

Adult vs Pediatric ICU Inspired oxygen fraction changes

◦ Size of FiO2 change (0.1 vs. 0.05)◦ SpO2 ranges: <88, 88-93, >93%; PaO2 ranges: <55, 56-68, >68 Torr

pH ranges◦ Adult: >7.45, 7.30 – 7.45, 7.15 – 7.29, <7.15◦ Pediatric: >7.45, 7.34 – 7.45, 7.25 – 7.34, 7.15 – 7.24, <7.15

Body weight for calculating tidal volume◦ Adult: predicted BW (obesity, BW calculated from height & gender)◦ Pediatric: actual BW (obesity & FTT, contractures; now formula for height from ulnar

length )

Tidal Volume (VT exhaled) measurement◦ Adult: measured at ventilator – use SET volume◦ Pediatrics: should be measured at ETT

Mode of Ventilation◦ ARDSNet – volume controlled◦ Pediatric – Pressure control - PC or PRVC (volume targeted)◦ Evolution in thinking re HFOV mode

Mechanical ventilation Frequently used intervention in ICU

◦ Care of patients on mechanical ventilator was a motivating factor in the development of ICUs (Watson, R.S. & Hartman, M.E. (2009). Epidemiology of Critical Illness. In D.S. Wheeler et al. (eds). Science and Practice of Pediatric Critical Care Medicine. London: Springer-Verlag.)

◦ Primary treatment for respiratory failure (ARDS/ALI). ◦ Also a common intervention for other conditions.◦ 20-64% (mean 30%) of PICU children require mechanical ventilation for some portion of their

stay◦ A common outcome measure in pediatric trials (vent free days, days on ventilator, etc.)

Labor intensive, accounts for disproportionate amount of resource usage and costs◦ 12% of overall hospital costs (Wunsch, H., Linde-Zwirble, W.T., Angus, D.C. et al. (2010). The epidemiology of mechanical ventilation use

in the United States. Crit Care Med, 38 (10), 1947-53)

Although life saving, mechanical ventilation has inherent risks (Bezzant & Mortensen, 1994; Newth et al., 2014)

◦ Oxygen toxicity◦ Barotrauma, pneumothorax, damage to lung tissue from excessive pressure, volume, and flow◦ Complications from intubation (tracheal damage)◦ Ventilator associated pneumonia◦ Dangers from drugs, stress; nutritional problems◦ Discomfort, pain, distress

High volume, High cost, High risk

Need for MV protocolsHeterogeneous patient characteristics

Many possible causes of ALI/ARDS

Best/optimal practices are not well understood

MV protocolsReported benefits of MV protocols in adult ICUs include decreased duration and costs of mechanical ventilation and improved collaboration between health care team members

Variable results in peds Schultz et al (2001) showed reduced time to extubation Randolph et al (2002) showed no decrease in weaning time. ◦ two complex paper-based protocol arms with poor

compliance in each arm

Both of these studies were limited to the weaning phase alone.

Mechanical Ventilation Course

End timeExtubation

MV course

Acute phase WeaningStable

Intubation

Intubationcriteria

Stabilizationcriteria

WeaningReadiness

test

ExtubationReadiness

test

Weaning extubation

failure

Extubationcriteria

End NIV criteria

NIV criteria

Definitions

WeaningStabilization “Routine management”

Variable results in peds Variability occurs throughout the entire length of ventilator management, not just the weaning phase.◦ Restepro et al (2004) found reduced time to spontaneous

breathing but no difference in overall ventilator duration. ◦ That study used a paper protocol to manage the overall

course of ventilation, but the authors noted as a limitation their inability to determine compliance with the protocol.

MV research in peds Willson et al. used a paper protocol outlining a broadly defined lung protective strategy.

Curley et al. used the adult ARDSNet ventilation paper protocol. ◦ Neither manuscript addressed protocol compliance.◦ Neither protocol was explicit

Jouvet et al.17 used a closed loop protocol for mechanical ventilation, but provided no details regarding its derivation from an adult protocol .

Intensive Care Medicine, 2009

Providers say they adopted ARDSnet lung protective ventilation – but we saw high variability in practices (single center)

MV in CPCCRN• Early CPCCRN ideas about trials of different modes of ventilation…

• Led to discussions of outcomes, paper on weaning and extubation, recognition that reducing practice variation can improve signal-to-noise (both for studies of MV; and for studies in which MV is a surrogate outcome.)

•This led to thinking about how MV decisions are made in the PICU … which led to our R21 grant

MV Protocol - Adult ARDSnet studies – most sites used a paper based protocol

Intermountain: Tom East, Alan Morris, and colleagues developed an explicit, computer-based protocol for mechanical ventilation, for care of adult ICU patients with ALI/ARDS

MV purposes (simplified)

1. Increase oxygen level

2. Reduce CO2 (reflected in the pH)

Protocol has rule sets for each (oxygenation and “ventilation”)

Different sets for different MODE and patient state

Translation for Pediatrics CPCCRN investigators recommended changes to adult protocol that they believed were necessary for practitioners at their site to accept the protocol recommendations.

Most of the changes were a matter of granularity (size)

* We also planned to update infrastructure from VB to Java/Drools

eProtocol MV – Peds KE challenges Complexityomultiple modes (VC, PRVC, PC, HFOV, extubated)o rules evaluating timing of ABG

◦ Non-invasive (O2 sat) vs invasive (pO2) measures

Large rule seto2824 “main” rules, plus othersoDROOLS file 56,410 lines

Little evidence of “best” or optimal practices◦ e.g., increasing PEEP vs increasing FiO2 for low oxygenation

R21 1. Examine usual care ventilator management practices

◦ compared to what the protocol would have recommended◦ Premise – when usual care and protocol are similar, the rule is probably going to

be seen by clinicians as acceptable◦ When different – either rule needs to be examined, OR this is an opportunity to

improve care◦ Prospective observational study – 8 hospitals

2. Examine issues of granularity – larger versus smaller changes; is this affected by large versus small child? And issues of potential acceptability of computer protocol• Survey with 50 fabricated scenarios. ICU attending MD and fellows.• Included attitude questions from UTAUT• Chose survey approach to focus on content (rules/recommendations), rather

than delivery method (CDS)

Δ N 2449

%

No Δ 1564 63.9%

↑ 363 14.8%

↓ 561 23%

↑↓ 39 1.6%

Actual Changes Madeto PIP or VR (ventilation)

Stratified actual changesby protocol bins…comparedwhat was doneto what protocol would haverecommended.

Single Institution

PC mode MV

N = 1484 ALI/ARDS

Heat map: correspondence between usual care and protocol rules

PEEP/FiO2 relationship

FiO

2

PEEP

We found WIDE variability in usual care practices within sites

PEEP/FIO2 Data – 8 CPCCRN PICUs

ALI2012

120Patients

3894ventilatorchanges

We found WIDE variability in usual care practices across sites

Set and forget?

Little change in modeacross course of care

Within a mode – themost frequent decisionwas to NOT CHANGEany setting.

Aim 2What are providers WILLING to do, at least in the context of a research study?

Even though they might not make changes to settings in usual care – are they willing to make changes if asked to do so?

Are they willing to have the research protocol communicated by means of a computer protocol (or are they resistant to the idea of using computer protocol)?

UTAUT constructs Performance expectancy

Effort expectancy

Attitudes toward technology

Social influence

Facilitating conditions

Anxiety

Self-efficacy

63.7% agree/strongly agree “using a computer protocol for ventilator management is a good idea”

and 68% agree that they have the knowledge necessary to use acomputer protocol

But want help files or a person who can assist with learning to use the protocol

Granularity, thresholds, modes

About evenly split (36.4%, 38.8%) on whether evaluation for ventilator change should occur every 2 or every 4 hrs

48% ventilate to the weight on admission to the PICU, 35.8% use predicted body weight

Wide range of responses as to what OI would trigger change from conventional to HFOV mode

Aim 1 showed VC/AC mode was rare in our sites but 71.9% say they use this mode in their PICU

Acceptability Overall ~ 80% accepted

Higher acceptance for FiO2 instructions (> 95%)

Acceptance rate varied by mode (highest acceptance for HFOV recommendations)

Recommendations “like protocol” were accepted at a higher rate than recommendations “not like” protocol – except lower acceptance for PEEP instructions

◦ Prefer to adjust FiO2 rather than PEEP to support oxygenation?◦ Clinicians seem uncomfortable with how high the protocol might push pressures (PEEP

and PIP)

Clinicians WILL decline “poor” recommendations (they stay vigilant)

eVentilator – Pediatric Version, May 2013

New CDS tool: Java/Drools

Architecture Core

Domain specific extension to core

GUI, IE, knowledge base, pt data etc. are separated

FDA IDE FDA considers this type of software a “device”: software that…accepts clinical findings … and generates recommendations for treatment…

FDA may or may not choose to require an IDE ◦ If so this requires extensive

documentation and several types of validation

FDA-IDE Working with colleagues in a U of Utah basic science lab for technical and safety evaluation of the CDS tool (e.g., run the software over extended time and watch for “crashes”), initial evaluations of usability, initial estimates of impact on workflow…

(naïve users; willing to enter data hourly 24x7)

Future Directions Infrastructure updates

◦ model driven architecture◦ JFX – GUI authoring

Other delivery platforms for multi-site research Web based? Mobile?

Map the terminology; represent knowledge according to standards – Health eDecisions format?

Alvin Feinstein, MD, 1977 (Yale)