1
Work Plan for Testing the LIS and EHR Systems• Define Test Flow based from Work Flow• Define a testing methodology• Develop high-level requirements for LIS Test Harness and LIS Test Tool• Identify conformance requirements external to machine-processable XML
message profile (on-going)– To include conditionals and variations in profiles (e.g., OIDs/no OIDs)– Include challenging cases (e.g., parent/child)
• Determine an initial set of use cases for testing (on-going)– Select from in-scope tests– Vocabulary subgroup identified 137 tests that accounts for 99% of lab results– Select core set of 5-10 (Start with one to establish template)
• Develop test cases– Test Data (Elincs/vendor test plans?; vendor data; NIST data for ELR IG (2007) and PH IG (2010)– Create Test Messages– Variation to core messages
• Create coverage analysis mapping test case coverage to conformance requirements– Continue until all conformance requirements handled
• Organize into a test plans (start with the EHR test plan)
2
Proposed Approach for Testing the EHR System
LISTest
Harness
EHR RI(System under Test)
ORU R01 Message
1. The EHR system is required to consume lab results messages that conform to the referenced standards and “incorporate” the data into their workflow.
2. The LAB system is a test tool and in this layout is a test harness. 3. The test harness will maintain a collection of pre-defined test cases that will include a range
of conformant and non-conformant messages covering various lab results and scenarios. The tester will also have the option to edit the test messages.
4. The tool sends LRI (sender profile) messages to the EHR (SUT).5. The EHR is expected to consume the message (as specified by the LRI receiver profile
specification) and “incorporate” into their system6. The EHR can be tested using a combination of inspection testing and automated testing. The
tests are designed to verify to a reasonable degree of certainty that the EHR correctly “incorporated” the lab results. For the most part this will be black box testing.
7. The Lab test tool will generate a juror document for inspection testing and a validation context for automated testing (of the acknowledgement message).
LR
IS
end
er
LR
IR
ece
ive
r
ACK R01 Message
Inspectiontesting
automatedtesting
3
EHR Validation Test Workflow I
Develop Specific Use Case Scenarios
Create Test Data
• Select from in-scope tests (small core set)• Typical and high priority• For each create relevant lab results data
• Document (e.g., in Excel)• Export to word document (resembles lab data sheet)• Add messaging information and map to HL7 (e.g., Excel)
Create/Generate Message• Create Message Profiles (use MWB)• Generate Messages (Message Profile + Test Data)
Verify Message• Verify that the message is correct according to the test case (e.g., contains correct LONIC code; missing element)• This is more than validation of the message
Create Test Validation Artifacts• Generate a juror document for inspection testing• Generate validation context file for automated testing
Send Test Message to EHR SUT• Configure message (e.g., Sending Application ID)• Configure test system for routing information• Send message to EHR SUT using MLLP
*
* The core set of messages can be modified to target specific test points
4
EHR Validation Test Workflow II
Receive/Validation Acknowledge Message
Perform Inspection Testing
• Receive acknowledgement message• Validate using message profile and validation context file• Generate Report
• Tester will instruct vendor to demonstrate incorporate of lab results into EHR system (compare w/ juror document)• Tester will use tool to manually add test outcome
Combine Validation Reports• Combine automated and inspection testing results• Display/Save/Print reports
End
5
Questions/Concerns (for testing the EHR)
• What assumptions can we make about the EHR system?– What is the EHR required to display?
• Is it sufficient to adequately assess the system?• What does CLIA buy us? • How do we make the link between what we sent and how they have incorporated it into their system?• E.g., the EHR will probably not display a LONIC code; it may map the code to its internal code
(representation) and display only the internal description as a text string—what is reasonable to accept?• What are other things to look out for?
– Could leverage other MU requirements (e.g., create CDA or send Lab results message to public health)
• What are the elements that need to be inspected/verified?– i.e., what needs to be in the juror document– How can we ensure that every element specified in the IG is handled correctly in the
EHR?– Is it practical to assess each required element without automated testing? If so how?
• What can be validated automatically?• How do we test the incorporation of vocabulary?
– Inspection testing– View configuration files?
6
Next Steps
• Develop Test Plans– LIS– EHR (initial focus)
• Test Plans to include– Work Flow– Test Flow and methodology– Test Cases
• Test Data (creation of a subgroup)
– Conformance requirements and coverage analysis appendix• Dimensions (OIDs, no OIDs, minimally populated, maximally populated, invalid, etc)
• Develop high-level design architecture/requirements– LIS Test Tool– LIS Test Harness
• Note: tool development discussion will not be part of this subgroup– We’ll have periodic tool demonstration and review