BENEFITS OF CONFORMANCE AND INTEROPERABILITY TESTING … · 2012. 2. 8. · Types of Testing Conformance testing • Testing a single part of a product for its compliance to a base

BENEFITS OF CONFORMANCE AND INTEROPERABILITY TESTING

PERSPECTIVES FOR 2012

Francois Fischer, Project Manager, ERTICO

Sebastian Mϋller, Technical Coordinator, ETSI

Why Produce Test Specifications?

A proven way to ensure interoperability and

standard compliance of ITS implementations

Their development provides a first validation

of the base standard(s)of the base standard(s)

Leads to significant improvements of base

specifications

Provide a basis for certification of products or

a test plan for an interoperability event

BPI: Acheiving Interoperability2

Types of Testing

Conformance testing

• Testing a single part of a product for its compliance to a base standard

• See Demo of Conformance Validation FrameworkInteroperability testingInteroperability testing

• Checking end-to-end functionality between a collection of equipment or devices

Interoperability testing with conformance checking

• Testing interoperability while monitoring conformance parameters

• This was applied for 1. ITS CMS PlugtestBPI: Acheiving Interoperability3

What is a Plugtests event?

A test event organized and run by a neutral body

• Scope, test infrastructure and test scenarios based on standard• Scheduling• Test Results and Feedback to Standards Development

An opportunity for engineers

• Evaluate the interoperability of their products• Evaluate the interoperability of their products• Validate their understanding of the base specification• Save time

An opportunity for vendors

• To demonstrate end 2 end interoperability to operators/end customers• Promote the technology and community

An opportunity for Standards Development

• Gaps, ambiguities, interpretations• A tool to validate and enhance the quality of standards

4

Interoperability Test Procedure

Connect two or three devices from different vendors

Check connectivity between devicesCheck connectivity between devices

Perform tests according to Plugtest Guide

• Check if devices can send/receive frames from each other• Check if data is handled correctly in the network and facility layers• Check if implemented algorithms work correctly

Result determination and reporting

• Result OK: run next test• Result NOK: check monitor tools to identify source of error• Report results in ETSI Test Reporting Tool

5

First ITS CMS Plugtest

Jointly organized by ETSI and ERTICO

Hosted by TNO

Sponsored by EC

Test specification produced by ETSITest specification produced by ETSI

• Distributed 2 months prior to event• Total of 33 tests• Developed in cooperation with DRIVE-C2X and EcoMove

ETSI – CTI Tools

• WIKI• Scheduling Tool• Test Reporting Tool

6

Participants

# Implementations

1 Cohda /NXP

2 CTAG

3 DENSO

4 Hitachi

5 Honda

# Tools

1 Fokus

2 TNO

3 TestingTech

4 Vector

7

Honda

6 ITRI

7 NEC

8 NordSys

9 Peek /TNO

10 QUWIC

11 Q-Free

12 Siemens

13 Vector

14 VTT

Pictures

Test session Test planning

Wiki

© 2010 – Proprietary Information of ETSI8

Debriefing

Results

Benefit for DRIVE-C2X

DRIVE-C2X designed a “Reference System” ITS station based on the ETSI TC ITS standards:• The “Reference System” will be then integrated in Vehicle or Roadside ITS

stations by different project partners

• DRIVE-C2X partners performed integration tests with “Reference System”. DRIVE-C2X specific challenges:• SW components developed by different partners, then ported to different HW

platforms and deployed on roadside infrastructureplatforms and deployed on roadside infrastructure

The interoperability tests performed during the Plugtests event made the project partners confident on the quality of the “Reference System”:• Successful test sessions executed between DRIVE-C2X partners – minor issues

were solved

• Test results confirmed that the DRIVE-C2X “Reference System” is compatible with other vendor implementations as well as the EcoMove project implementations

• The results prove the good quality of the reference ETSI TC ITS base standards

© ETSI 2011. All rights reserved9

The ITS interoperability

workshop

Aimed to disseminate the experience and lessons learned during the Plugtests

Presentations from:• DRIVE-C2X, ERTICO, ETSI, Fraunhofer FOKUS, Renault, SISCOGA,

TNO, VECTOR

Round table about the “future steps”:Round table about the “future steps”:• With the participation of Honda, InnovITS, Renault and TNO

• Why deployment requires Interoperability• How Plugtests events can accelerate standardization• Conformance testing and Plugtests event to pave the way for

certification

TNO demo on cooperative ITS network emulator


Pictures

2nd ITS CMS Plugtests event

11 – 15 June 2012

Hosted at IFSTTAR, Versailles, France

• http://www.ifsttar.fr/presentation/implantations/versailles-satory/Scope

• optimized CAM, DENM• SAM• SAM• TOPO, SPAT• All optional tests from 1st ITS CMS Plugtest

All facilities for lab test are available

• Faraday cages• Test Network• Central Position Server for GPS feeds

Anybody interested in this event please contact

[email protected] and [email protected]

12

Links

The Plugtests event:

• http://www.ertico.com/1st-interop-event-cms-14-18-nov-2011

• http://www.etsi.org/plugtests/ITS_CMS/Home.htm•

The Workshop

• http://www.ertico.com/1st-interop-event-workshop/• http://www.etsi.org/plugtests/ITS_CMS/Workshop.htm


Other ITS domains

Plugtests event are planned for other ITS

technologies:

• eCall• A Plugtests event is planned 21-25 May 2012 at • A Plugtests event is planned 21-25 May 2012 at

InnovITS ADVANCE at Nuneaton (Birmingham, UK)

• TPEG• A plugtests event is planned Q3 2012 at ERTICO in

Brussels


22 - 26 October 2012

Documents

BENEFITS OF CONFORMANCE AND INTEROPERABILITY TESTING … · 2012. 2. 8. · Types of Testing Conformance testing • Testing a single part of a product for its compliance to a base