The Connectathon:IHE’s Conformance Testing Process

Presented by:Presented by:

Mike Nusbaum & Mike GlickmanMike Nusbaum & Mike GlickmanIHE Connectathon & Interoperability Showcase Planning CommitteesIHE Connectathon & Interoperability Showcase Planning Committees

January 16January 16thth, 2007, 2007

2

“Connectathon 101”

3

Objective and Agenda

• What is the What is the ConnectathonConnectathon??• To provide a high-level understanding of the Connectathon To provide a high-level understanding of the Connectathon

processes, and their value within the context of IHEprocesses, and their value within the context of IHE• To describe what is currently happening on the To describe what is currently happening on the

Connectathon floorConnectathon floor• To prepare you for a tour of the Connectathon floor, To prepare you for a tour of the Connectathon floor,

immediately following this presentation…immediately following this presentation…

4

A Phenomenon…

5

Did you know…

• The first ConnectathonThe first Connectathon– held in 1999 in Chicagoheld in 1999 in Chicago– 23 vendors, 47 applications tested, 1 Integration Profile23 vendors, 47 applications tested, 1 Integration Profile

• The 2007 North American Connectathon boasts:The 2007 North American Connectathon boasts:– 77 vendors registered (up from 55 in 2006)77 vendors registered (up from 55 in 2006)– Over 150 individual applications being testedOver 150 individual applications being tested– Over 350 engineers working collaboratively to test interoperabilityOver 350 engineers working collaboratively to test interoperability– 52 integration profiles are being tested, many of which are new in 52 integration profiles are being tested, many of which are new in

20072007– 1000’s of vendor-vendor connections; 10,000’s of transactions1000’s of vendor-vendor connections; 10,000’s of transactions– Represents many 1000’s of hours of effort by sponsors, vendors, staffRepresents many 1000’s of hours of effort by sponsors, vendors, staff

• 51 Vendors applied to promote their success at the 2007 51 Vendors applied to promote their success at the 2007 Interoperability Showcase at HIMSSInteroperability Showcase at HIMSS

6

Timely access to Timely access to informationinformationTimely access to Timely access to informationinformation

Easy to integrate Easy to integrate productsproductsEasy to integrate Easy to integrate productsproducts

Products Products with IHEwith IHEProducts Products with IHEwith IHE

IHE IHE DemonstrationsDemonstrationsIHE IHE DemonstrationsDemonstrations

Develop Develop technical technical

specificationsspecifications

Develop Develop technical technical

specificationsspecifications

Proven Standards Adoption Process

Document Use Document Use Case Case RequirementsRequirements

Document Use Document Use Case Case RequirementsRequirements

Identify available Identify available standards (standards (e.g. HL7, e.g. HL7, DICOM, IETF, OASIS)DICOM, IETF, OASIS)

Identify available Identify available standards (standards (e.g. HL7, e.g. HL7, DICOM, IETF, OASIS)DICOM, IETF, OASIS)

Testing at Testing at ConnectathonsConnectathonsTesting at Testing at ConnectathonsConnectathons

7

Scenes from Connectathon 2006

9

Encouraging Vendors toImplement IHE Profiles

• Connectathon participation is open to all “software Connectathon participation is open to all “software developers”, vendors, open source, providersdevelopers”, vendors, open source, providers

• Interoperability Specifications (IHE Profiles) are set before Interoperability Specifications (IHE Profiles) are set before users issue call for participation with deadline users issue call for participation with deadline (applicants have (applicants have to bid)to bid)

• Vendors that implement IHE profiles assume strategic and Vendors that implement IHE profiles assume strategic and marketing advantagesmarketing advantages

• Vendors who pass the IHE Connectathon are given the Vendors who pass the IHE Connectathon are given the opportunity to demonstrate IHE capabilities during major opportunity to demonstrate IHE capabilities during major conferences (HIMSS, ACC, RSNA, etc.)conferences (HIMSS, ACC, RSNA, etc.)

Many “carrots”, but only one stick:Many “carrots”, but only one stick:

If you don’t play, you can’t pass!!If you don’t play, you can’t pass!!

10

IHE Testing Process

UsersUsers

Sponsors:Sponsors:ProjectProjectManagementManagementTeamTeam

VendorsVendors

Sponsors:Sponsors:ExhibitsExhibits

DevelopDevelopTesting ToolsTesting Tools

Testing ToolsTesting Tools

ImplementImplementProfile ActorsProfile Actors

In-HouseIn-HouseTestingTesting

ConnectathonConnectathon

DemonstrationDemonstration

DeployDeploySystemsSystems

Testing ResultsTesting Results

ApprovesApprovesTest LogsTest Logs

IHE Technical Framework (IHE Profile Specifications)

Product +Product +IntegrationIntegrationStatementStatement

11

What is a Connectathon?

Cross-vendor, live, supervised, structured testsCross-vendor, live, supervised, structured tests

• All participating vendors’ products tested together in the All participating vendors’ products tested together in the same place/timesame place/time

• Experts from each vendor available for immediate problem Experts from each vendor available for immediate problem resolution… fixes are done in minutes, not months!!resolution… fixes are done in minutes, not months!!

• Each vendor tests with multiple trading partners (actual Each vendor tests with multiple trading partners (actual product to actual product)product to actual product)

• Testing of real-world clinical scenarios using IHE Testing of real-world clinical scenarios using IHE Integration ProfilesIntegration Profiles

12

Connectathon Testing is based on specifications laid out in the Technical Framework

Technical FrameworkTechnical Framework

Part 1Part 1: : Integration Profiles Integration Profiles model the business process model the business process problem (use case) and its problem (use case) and its solution.solution.

Part 2Part 2: : Transactions Transactions define in how current define in how current standards are used to solve standards are used to solve the business problem the business problem defined in the Integrationdefined in the IntegrationProfiles.Profiles.

ConnectathonConnectathon: : Vendors Vendors register to test their product register to test their product as an actor(s) within an as an actor(s) within an Integration ProfileIntegration Profile

13

Connectathon: Managed Process

• Structured testing supervised by Technical Project Structured testing supervised by Technical Project Management team (34 PM’s and Monitors)Management team (34 PM’s and Monitors)

• Real-time “dashboard” indicating tests in progressReal-time “dashboard” indicating tests in progress• Successful results recorded and available using Successful results recorded and available using

automated toolsautomated tools• Successful results published by sponsorsSuccessful results published by sponsors

•National Institute of Standards and Technology National Institute of Standards and Technology (NIST)(NIST)•Mallinckrodt Institute of Radiology, Washington Mallinckrodt Institute of Radiology, Washington UniversityUniversity•Northwestern University Medical SchoolNorthwestern University Medical School•University of Maryland Health SystemUniversity of Maryland Health System•University Health Network (Canada)University Health Network (Canada)•Carilion Health SystemCarilion Health System•HITSPHITSP•Montefiore Medical CenterMontefiore Medical Center•IEEE (Institute of Electrical and Electronics IEEE (Institute of Electrical and Electronics Engineers, Inc.)Engineers, Inc.)

•OFFIS (Oldenburger Forschungs- und OFFIS (Oldenburger Forschungs- und Entwicklungsinstitut für Entwicklungsinstitut für •Informatik-Werkzeuge und -Systeme, University of Informatik-Werkzeuge und -Systeme, University of Mainz, Germany)Mainz, Germany)•McGill UniversityMcGill University•University of QuebecUniversity of Quebec•Partners HelathcarePartners Helathcare•National Institute for Research on Informatics and National Institute for Research on Informatics and Automatism INRIAAutomatism INRIA•SSHA (Smart Systems for Health Agency)SSHA (Smart Systems for Health Agency)•Elizabeth Wende Breast ClinicElizabeth Wende Breast Clinic•Industry ConsultantsIndustry Consultants

14

“Real-Time” Connectathon Results

•Tool contains Connectathon results from 2001 to present

•Part of the “KUDU” project management tool, developed in part by IHE Europe

•Development underway to build a new PRODUCT REGISTRY, containing specific vendors’ product implementations of IHE Integration Profiles

15

Connectathons are GLOBAL and occur throughout the year

• IHE North AmericaIHE North America– January 2006January 2006

• IHE AsiaIHE Asia– February 2006 February 2006

• IHE Europe IHE Europe – April 2006April 2006

• IHE North AmericaIHE North America– October 2006October 2006

• IHE North America IHE North America (USA and Canada)(USA and Canada)

– January 2007January 2007• IHE AsiaIHE Asia

– February 2007February 2007

• IHE EuropeIHE Europe– April 2007April 2007

• ……....

16

MESA Testing Tools

““Medical Environment Simulators and Analyzers” Medical Environment Simulators and Analyzers”

• Software and documentation distributed to vendors Software and documentation distributed to vendors participating in the testing processparticipating in the testing process

• Each test performs functional testing of a single Each test performs functional testing of a single actor, in a specific profile, by simulating remaining actor, in a specific profile, by simulating remaining actorsactors

• MESA tests must be successfully completed before MESA tests must be successfully completed before the Connectathonthe Connectathon

17

MESA Testing Tools

• Primary developer: MIR, University of WashingtonPrimary developer: MIR, University of Washington• Other contributors:Other contributors:

– NIST, US Dept of Commerce: XDS toolsNIST, US Dept of Commerce: XDS tools– OFFIS, Germany: DICOM ScopeOFFIS, Germany: DICOM Scope– Canada Health Infoway, Canada: XDS-I, PIX/PDQ/HL7v3Canada Health Infoway, Canada: XDS-I, PIX/PDQ/HL7v3– INRIA, France: Lab HL7v2.5INRIA, France: Lab HL7v2.5

• To be replaced by GAZELLE: a new testing tool architected To be replaced by GAZELLE: a new testing tool architected to facilitate distributed testingto facilitate distributed testing

18

Gazelle Project

• Joint project of IHE North America, IHE Europe and IHE Joint project of IHE North America, IHE Europe and IHE JapanJapan– Additional partners invited to take partAdditional partners invited to take part– Steering Committee populated by sponsoring organizationsSteering Committee populated by sponsoring organizations

• Evolution of current Kudu testing management toolsEvolution of current Kudu testing management tools• Will enable testing on demandWill enable testing on demand

– Acceptance testing by institutionsAcceptance testing by institutions– Internal testing by vendorsInternal testing by vendors– Vendor-to-vendor remote testing via the InternetVendor-to-vendor remote testing via the Internet

• ““Virtualized” Connectathon testing to augment the face-to-Virtualized” Connectathon testing to augment the face-to-face testing eventface testing event

19

Connectathon Scorecard

20

What happens after the Connectathon?

• Successful results (specific by IHE profile/actor) are Successful results (specific by IHE profile/actor) are published by the sponsors published by the sponsors (www.ihe.net)(www.ihe.net)

• Vendors self-certify, by publishing Vendors self-certify, by publishing IHE Integration IHE Integration StatementsStatements: Precise and explicit public interoperability : Precise and explicit public interoperability commitment for a specific commercial product.commitment for a specific commercial product.

• Only vendors who are successful are entitled to participate in Only vendors who are successful are entitled to participate in Interoperability Showcase demonstrations:Interoperability Showcase demonstrations:– HIMSS Annual Conference (February 26HIMSS Annual Conference (February 26thth – March 1 – March 1stst, New Orleans), New Orleans)– ACC Annual Conference (March 24ACC Annual Conference (March 24thth – 27 – 27thth, New Orleans), New Orleans)– Canada e-Health 2007 Conference (May 27Canada e-Health 2007 Conference (May 27 thth – 30 – 30thth, Quebec City), Quebec City)– RSNA Annual Conference (November 25RSNA Annual Conference (November 25thth – 30 – 30thth, Chicago), Chicago)– others…others…

21

IHE Integration Statement

22

RFPs & Integration Statements

• Be Brief ?Be Brief ?– ““The system must support HL7”The system must support HL7”

• Be Effective ?Be Effective ?– ““The system must support the following HL7 V2 messages The system must support the following HL7 V2 messages

according to the following 100 pages of specifications”according to the following 100 pages of specifications”

• Be Both:Be Both:– ““The system must support IHE Patient ID Cross-reference as a The system must support IHE Patient ID Cross-reference as a

Patient Identifier Source actor”Patient Identifier Source actor”

• Integration StatementIntegration Statement– Version 2.1 of the ACME Enterprise HIS supports IHE Patient ID Version 2.1 of the ACME Enterprise HIS supports IHE Patient ID

Cross-referencing as Patient Identifier Source actorCross-referencing as Patient Identifier Source actor

Requiring vendors to publish their products’ Requiring vendors to publish their products’ IHE Integration IHE Integration Statements Statements provides a very effective catalyst in achieving provides a very effective catalyst in achieving

market-driven interoperabilitymarket-driven interoperability

23

RFP Language (actual example)

24

Connectathon Tour

• You will be led by “docents”You will be led by “docents”• There will be a number of “stopping points”, illustrating There will be a number of “stopping points”, illustrating

different aspects of the Connectathondifferent aspects of the Connectathon– VendorsVendors describing “what is going on now” describing “what is going on now”– VendorsVendors describing “what happens after the Connectathon is over” describing “what happens after the Connectathon is over”– MonitorsMonitors, describing how tests are evaluated and scored, describing how tests are evaluated and scored– Project ManagersProject Managers, describing how the Connectathon is organized , describing how the Connectathon is organized

and managedand managed

• Staggered departure, in order to minimize disruptionStaggered departure, in order to minimize disruption• Feel free to observe and listen, but respect the effort that is Feel free to observe and listen, but respect the effort that is

underway and refrain from disrupting the engineers at underway and refrain from disrupting the engineers at work! work!

25

Connectathon Floor Layout

Enjoy the Tour!Enjoy the Tour!Please reconvene at 2:15pmPlease reconvene at 2:15pm