Upload
adrian-walsh
View
212
Download
0
Tags:
Embed Size (px)
Citation preview
IHE International Meeting
Gazelle ProjectGazelle ProjectSteve Moore, MIR Steve Moore, MIR
Eric Poiseau, INRIAEric Poiseau, INRIA
IntroductionIntroduction
IHE is expanding
New domains New participating countriesMore companies involved
Need better testing process
Need for conformity testing Need for interoperability
4
Participation to the European Connect-a-thon Participation to the European Connect-a-thon
2001 2002 2003 2004 2005 2006 20070
50
100
150
200
250
300
11
3343 46
7563
0
18
57
74 79
99
120
0
35
100
135
180
250
280
0
DomainsIntegration profilActorsCompaniesSystemsParticipants
What we have learnedWhat we have learned
Our Experience :
8 years of connect-a-thon8 years of MESA toolsConnect-a-thon on 3 continents !
Need to distinguish between :
Conformance testing Interoperability testing
Objectives of the projectObjectives of the project
Provide IHE with a new set of test tools :
For more in depth testing during IHE connect-a-thon
For companies internal testingFor testing between companiesFor Healthcare Enterprise QA/Acceptance
tests
IHE International Meeting
Conformance TestingConformance Testing
Conformance Testing (1/2)Conformance Testing (1/2)
Is unit testing Tests a single ‘part’ of a device
Tests against well-specified requirements For conformance to the requirements specified in the
TF and the referenced standards Usually limited to one requirement per test.
Tests at a 'low' level At the protocol (message/behaviour) level.
Requires a test system (and executable test cases) Can be expensive, tests performed under ideal
conditions
Conformance Testing (2/2)Conformance Testing (2/2)
High control and observability
Means we can explicitly test error behaviour Can provoke and test non-normal (but legitimate)
scenarios Can be extended to include robustness tests
Can be automated and tests are repeatable
Conformance Testing is DEEP and NARROW
Thorough and accurate but limited in scope Gives a high-level of confidence that key components
of a device or system are working as they were specified and designed to do
Limitations of Conformance TestingLimitations of Conformance Testing
Does not prove end-to-end functionality (interoperability) between communicating systems
Conformance tested implementations may still not interoperate• This is often a specification problem rather than a testing
problem! Need minimum requirements or profilesDoes not test a complete system
Tests individual system components, not the whole• A system is often greater than the sum of its parts!• Does not test functionality
Does not test the user’s ‘perception’ of the systemStandardised conformance tests do not include proprietary ‘aspects’
Though this may well be done by a manufacturer with own conformance tests for proprietary requirements
IHE International Meeting
Interoperability TestingInteroperability Testing
Interoperability TestingInteroperability Testing
Is system testing
Tests a complete device or a collection of devicesShows that (two) devices interoperate
within a limited scenario !Tests at a ‘high’ level (as perceived by users)
Tests the ‘whole’, not the parts Tests functionality
Does not necessarily require a test system
Uses existing interfaces (standard/proprietary)Interoperability Testing is BROAD and SHALLOW
Less thorough but wide in scope Gives a high-level of confidence that devices (or components in a
system) will interoperate with other devices (components)
Limitations of Interoperability TestingLimitations of Interoperability Testing
Does not prove interoperability with other implementations with which no testing has been done
A may interoperate with B and B may interoperate with C. But it doesn’t necessarily follow that A will interoperate with C.
Combinatorial explosionDoes not prove that a device is conformant
Interoperable devices may still interoperate even though they are non-conformant
Cannot explicitly test error behaviour or unusual scenarios
Or other conditions that may need to be forced (lack of controllability)
Has limited coverage (does not fully exercise the device)Not usually automated and may not be repeatable
IHE International Meeting
Conformance or Conformance or Interoperability Testing ?Interoperability Testing ?
Conformance or InteroperabilityConformance or Interoperability
Both are Needed :
Complementary, not competitiveETSI : « While it is not absolutely necessary to
undertake both types of testing, the combined application of both techniques gives a greatly increased confidence in the tested product and its chances of interoperating with the other similar products
ETSI : European Telecommunications Standards Institute
IHE International Meeting
Gazelle Year 1 Gazelle Year 1 PlanPlan
Year 1 targetYear 1 target
Prototype evaluate the design measure our estimatesApplied to selected profiles but designed to work
for all (known ones)• Radiology Scheduled Workflow (SWF)• Laboratory Scheduled Workflow (LSWF)• Patient Identity Cross-referencing (PIX)• Audit Trail and Node Authentication (ATNA)
Year 1 development planYear 1 development plan1st Quarter
HL7 & Dicom conformance tools (selected profiles) Integration Statement Management Application Logging Subsystem
2nd Quarter
Web system for Interoperability and conformance Tests HL7 & Dicom data sets
3rd Quarter
Simulated Actors User Management
4th Quarter
Report and evaluation
Year 1 ResourcesYear 1 Resources
IHE Europe 2 FTE1 FTE Inria
IHE North America 2 FTE (RSNA-MIR) NIST (XDS) ? IHE Canada / Infoway ?
IHE Asia
IHE-Japan (Translation work, Spec review)
ConsequencesConsequences
Mesa Tools
NO NEW DEVELOPMENT LIMITED SUPPORT BY STEVE
Kudu
NO NEW DEVELOPMENT LIMITED SUPPORT BY ERIC
But Connectathon Management Continues !
IHE International Meeting
Questions ?Questions ?