Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
C M S A L L I A N C E T O M O D E R N I Z E H E A L T H C A R EFebruary 2017
Bonnie:An Electronic Clinical Quality Measure Testing Tool
Lizzie CharbonneauThe MITRE Corporation
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
What is Bonnie?
Bonnie is a clinical quality measure testing tool that allows:
– Loading electronic Clinical Quality Measures (eCQM) from the Health Quality Measure Format (HQMF) format
– Exploring the behavior and complexity characteristics of clinical quality measures
– Rapidly building synthetic patients using data elements defined as part of the measure definition
– Testing synthetic patients against existing and updated versions of a clinical quality measure
– Creating synthetic patients for use in certification with Cypress
– Sharing synthetic patient records with other Bonnie users using the patient bank
2
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved. 3
Why Bonnie?
Prior to Bonnie, the only way to test electronic clinical quality measures (eCQM) was by hand.
– eCQMs are large and complex.
– Checking by hand is time consuming and error prone.
Before Bonnie was in use, 381 logic errors were reported by implementers. Each measure had at least one issue.
Bonnie was introduced to enable automated testing of eCQMsprior to publication.
After Bonnie was introduced, the number of reported errors dropped by 95%.
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
History
First released April 2014
Used extensively for testing 2015 and 2016 annual update measures– 2015
Over 5,000 synthetic test patient records, 80% logic coverage
– 2016 Over 10,000 synthetic test patient records, 100% logic coverage
Currently being used for testing 2017 annual update measures
– Over 16,000 synthetic test patient records– Striving for 100% coverage of the logic
National Quality Forum (NQF) recommends use of Bonnie to support NQF eCQM endorsement
4
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Bonnie Usage over Time
5
5,000
10,000
15,000
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Users of Bonnie
Measure Developers
– As part of the EHR Incentive Program, Quality Payment Program, and other CMS Quality Reporting Programs (e.g., IQR, CPC+)
– NQF recommends use of Bonnie for NQF endorsement
EHR Vendors
– Get head start on integrating new measures into their systems
– Perform calculation validation
6
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Bonnie Screenshot
7
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Bonnie Screenshot (continued)
8
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Bonnie Is under Active Development
Current development work includes:
– Support for Clinical Quality Language (CQL)
Expect all Bonnie functionality including new features to be updated to support CQL
Recently completed development work includes:
– Measure History
– Patient Dashboard
– Value Set Debugging
– Excel Patient Export
– Bonnie Application Programming Interface (API)
– Quality Data Model (QDM) 4.3 Update
9
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Recent Features
10
API
Value Set Debugging
Excel Patient Export
Bonnie API
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
New Feature: Measure History
When measures are updated between annual update cycles, it can be difficult to keep track of the nuanced changes between measure versions.
Patient calculations often change across measure versions. It can be difficult to recall how a patient previously calculated and how they’re expected to calculate with the new measure version.
New measure history feature includes:
– A view of patient calculations across measure versions
– A view of how measure logic has changed across versions
– A detailed view of patient calculations across measure versions
This feature is currently in production.
11
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Measure History (continued)
12
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Measure History (continued)
13
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Measure History (continued)
14
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Measure History (continued)
15
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
New Feature: Patient Dashboard
User feedback indicated that patient test deck planning and editing is easier to do within the context of other patients.
New patient dashboard feature includes:
– Showing patient calculation results in the context of other patients
Includes aggregate results and individual logic clause results
– Enabling inline editing of patient metadata (name, expected values, birthdate and deathdate, etc.)
– Allowing users to easily access Patient Builder editor
This feature is currently in production.
16
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Patient Dashboard (continued)
17
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Patient Dashboard (continued)
18
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Patient Dashboard (continued)
19
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Support for the Clinical Quality Language (CQL)
The CQL is a new standard for trial use (STU) for describing eCQMs.
CQL logic is expected to eventually replace existing QDM logic.
Regression testing during the transition will be vital to ensure that measures continue to convey the same clinical intent.
Bonnie will support testing of both CQL-based and QDM-based measures during the transition.
The functionality that Bonnie implements will leverage the extensive existing open source CQL implementation.
Initial CQL support is available on the Bonnie staging server: https://bonnie.ahrqstg.org/
20
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Support for CQL (continued)
21
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Support for CQL (continued)
Upcoming Feature Improvements for CQL
– More readable logic display
– Logic highlighting
– More granular coverage and complexity calculations
– Patient Dashboard support for CQL
– Excel Export support for CQL
– Measure History support for CQL
22
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Bonnie User Group
Interaction with our user community is vital as we
– Seek feedback for new features under development
– Determine the future direction of the application
– Share news about updates and changes
The online forum for the Users Group is
– https://groups.google.com/forum/#!forum/bonnie
23
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Where to Look for Info
Main Bonnie site
– https://bonnie.healthit.gov/
JIRA for tracking issues
– https://oncprojectracking.healthit.gov/support/projects/BONNIE
Contact development team
Open Source on GitHub
– https://github.com/projecttacoma/bonnie
24
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Questions?
25
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Backup Slides
26
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Clinical Quality Measures
Health Care Reports
Measure provider delivery of standard of care to patients
Electronic specification: eCQM
Combine logical statements with clinical concepts
27
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Patient Populations
28
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
History
The process of Measure Development has been grueling:
– Does the measure logic capture clinical intent?
– Lots of subtleties: timing, operator precedence, etc.
– Hundreds or thousands of lines of logic
– Every release is manually reviewed
29
Approved for Public Release; Distribution Unlimited. 17-0467 © 2017, The MITRE Corporation. All Rights Reserved.
Final Notes
Open source
– http://bonnie.healthit.gov/
30