Remote Conformance & Interop Testing F-Interop · 2016. 10. 7. · 1 - TPAC2016 – WoTIG Meeting, 22nd September 2016 Remote Conformance & Interop Testing 1 F-Interop Online Platform

1 - TPAC2016 – WoT IG Meeting, 22nd September 2016

Remote Conformance & Interop Testing

1

F-InteropOnline Platform of Interoperability and

Performance Testsfor the Internet of Things

TPAC2016 – Web of Things IG Meeting – Lisbon22nd September 2016

César Viho & Federico Sismondi INRIA - France


FF-F-Interop H2020 Project! """#$%&'()*+,#)-! ./0+1)23)*/45.6/7 8./9:(+3)*/45.;! !"#"$%&'()!'&*%#+!"'%)$+)"'+),"*%&"*(-+$+,.'()!'&"*/%*0()1"',"2,',%%$2',%'23&&%*,'"0"*4+)4',"15)%$%4+"2'/*%0'*"2"(*15',%'2,()!(*!+6(,+%)'()!'0(*7",'$(3)15

!


Goals

1. Describe the F-Interop platform2. Is this useful for the WoT community?3. How the WoT community can help?! Introduce the F-Interop open call


! !"#$! >=1)/(&2)/='?/*)>+-*:)>! *-''&'@/:+?)/)=*AB! =::)A)*=()/>(='?=*?&C=(&+'/,*+:)>>

! !%&$'()*'+,-.()/0$! &'()*+,/()>(>/"&(D+-(/'))?&'@/(+/(*=1)A! A+")*/?)1)A+,2)'(/:+>(! $=>()*/?)1)A+,2)'(/+$/>(='?=*?>%3=>)?/,*+?-:(>

" 2+*)/>(='?=*?>%3=>)?/,*+?-:(>

Why Why remoteremote conformance & interop?


Core Idea

E%F'()*+,/G)*1)*

:A&)'(

H&2,A)2)'(=(&+'/-'?)*/()>(I

J+'$+*2=':)/K)>(&'@


Core Idea

E%F'()*+,/G)*1)*

:A&)'(

H&2,A)2)'(=(&+'/-'?)*/()>(I

F'()*+,/K)>(&'@

:A&)'(

H&2,A)2)'(=(&+'/-'?)*/()>(I


4

A. Simple test Device F-Interop test server.B. Deported test with downloaded resource.C. Interop test against FIRE+ device.D. Interop test against FIRE+ large set of devices.E. Interop test between two distinct devices provided by developers.F. Interop test against several devices remotely connected

by one or several developers only.G. Interop test between several devices remotely connected by

developers combined with FIRE+ devices, enabling instance largescale interop (“online plugtests”) and performance tests.

L# K)>()?/M)1&:)/#" E%F'()*+,/()>(/>)*1)*N# M),+*()?/()>(/"&(D/?+"'A+=?)?/*)>+-*:)J# O)2+()/&'()*+,/"&(D/4/,=*(&:&,='(>M# F'()*+,/=@=&'>(/()>(3)?P# Q+:=A/&'()*+,E# O)2+()/&'()*+,/"&(D/0/,=*(&:&,='(>R# O)2+()/&'()*+,/"&(D/0/,=*(&:&,='(>/='?/()>(3)?>

Different Configurations


! 10*2134&S"""#$)?T$&*)#)-U()>(3)?>V! 4T/()>(3)?>! W.555/'+?)>

! #)05(6S+')A=3#)-V! F':A-?)>/X/F+K%A=3/

?),A+B2)'(>/S&':A-?&'@/4Y4;/F+K/'+?)>V

! 3,7'8(6S"""#&+(A=3#)-V

84/()>(3)?>Z/TY66/'+?)>

Testbeds


Targeted Standards

! F'&(&=AAB/>(='?=*?>/+$/(D)/F+K/*)=A2! J+L[! XK&GJ\! XQ+][L0

! ])/(=^)Z/=>/=/>(=*(&'@/,+&'(Z/(D)/PKGF/,A-@()>(>/>,):&$&:=(&+'>/='?/3-&A?/='/=*:D&():(-*)/(D=(/=AA+">/(D+>)/(+/3)/?+')/*)2+()AB

! 9,):;/6:0)$/,)$'(;0'0>.0+:0*'6?'*0$/@)! F':A-?&'@_

! +')`4`! A06',B'7C/)@$'DA,7E


CoAPCoAP remote online interop testingremote online interop testingA proof of concept


Example Example CoAPCoAP Test! E*+2/PKGF/,A-@()>(/J+L[aTZ/FPKE;


Base Architecture (Base Architecture (CoAPCoAP interop)

P1)'(N->89(--+,:; -*%7"*<

+*:D)>(*=(+*

()>(/>-&()

")3

A+@@)*:A&

=@)'(

L`b[

J+L[ :A&)'(/SJ+,,)*V

(-'

E%F'()*+,/>)*1)*

->)*L

=@)'((-'

->)*N J+L[ >)*1)*

SJ=A&$+*'&-2V


Base Architecture (CoAP Base Architecture (CoAP interop demo)

P1)'(N->89(--+,:; -*%7"*<

+*:D)>(*=(+*

")3

A+@@)*:A&L`b[

E%F'()*+,/>)*1)*

->)*

=@)'(

J+L[ :A&)'(/SJ+,,)*V

(-'

()>(/>-&()

J+L[ >)*1)*SJ=A&$+*'&-2V

(-'


Download the Agent


Connect Connect to the Fto the F-to the F-Interop Server


Select and Select and Start Start the Tthe Test est CCase


Send Send CoAPCoAP Packets


Finish Finish Test Test CCase

19 - TPAC2016 – WoT IG Meeting, 22nd September 2016TPAC2016 – WoT IG Meeting, 22nd September 2016

Verd

ict!


Under the Hood: What’s a test?


Next Milestones

! c-AB/45.X! 2&'&2=A/J+L[/&'()*+,/()>(&'@/S?+')V/%d/>))/?)2+

! 0+1)23)*/45.X! E-':(&+'=A/,A=($+*2/=1=&A=3A)! J+L[/J9OP/&'()*+,/()>(>

! `=*:D/45.Y! XK&GJ\/>-,,+*(Z/-,?=()/=(/FPKE&+'V/

! c-AB/45.Y! e>)/=(/XK&GJ\UXA+/,A-@()>(>! -/)/-(8'A,7 /):0;,.':0$:/)@


WoTWoT interop test case example

G+-*:)_/D((,>_UU@&(D-3#:+2U"8:U"+(U3A+3U2=>()*U,A-@$)>(U45.X%3)&f&'@U,A-@$)>(%()>(%:=>)>%3)&f&'@%45.X#2?


How the How the WoTWoT community can help?

! 9,):;/6/)g()'?&'@ E%F'()*+,/$+*/&'()*+,/&'/]+K/:+'()g(! Q&>(/*)h-&*)2)'(>Z/&?)'(&$B/^)B/,*&+*&(B/]+K/>(='?=*?>! M)1)A+,/()>(/>-&()>/$+*/S')"V/>(='?=*?>! [*+1&?)/$))?3=:^/+'/=*:D&():(-*)/='?/:D+&:)>

! G$0;$F! e>)/E%F'()*+,/$+*/*)2+()/&'()*+,/)1)'(>U,A-@()>(>


#.0)'9(88


#.0)'9(88'9(:0@,;/0$

$ H0I':0$:/)@':,,8$'(+/)g()'?/:=,=3&A&(&)>/+$/E%F'()*+,

$ H0I':0$:'*0$+;/.:/,)$'(+/()>(/:+'$+*2=':)/='?/

&'()*+,)*=3&A&(B/+$/+(D)*/>(='?=*?>

$ !%&'1J3):0;,.'($$0$$-0):';0.,;:$F'G`P/?)1&:)/F'()*+,/

()>(> (+/()>(/E%F'()*+,/,A=($+*2

$ K8( )1)'(>


!/)g()'>&+'> 8 .55 555

H0I'/):0;,.':0$:'*0$+;/.:/,)$ 8 X5/555

G`P/?)1&:)>/E%F'()*+,/()>(>/='?/*),+*( .5 .5/555

K8


3-.,;:():'"(:0$

Formally launch Open Call & open the

application process

Open Call Starts

July

… 2016 … 2017

Close the application process

Delivery of plugtest events

25th January

2017

Conduct evaluations &

select 3rd parties

26thApril 2017

30thAugust

2017

Commence experiments/

projects

May

… 2018

Review 1st release of tools & tests projects

Projects kick off

AprilFebruary

2018

Results announcement

…


T,I':,'(..8?U

$ K)2,A=()/$+*/(D)/,*+,+>=A

$ R-&?)/$+*/L,,A&:='(>

$ G(='?=*?/F'?->(*&=A/Pg,)*&2)'(/J+'(*=:(

$ 9,)'/J=AA/K)*2>/='?/J+'?&(&+'>

$ !


KD='^ B+- $+*/B+-* =(()'(&+'

9,)'%:=AA_/D((,_UU"""#$%&'()*+,#)-U&'?)g#,D,U+,)'%:=AA

[A)=>)Z/$))A $*))/(+ :+'(=:(/->/?&*):(AB +*/A=()* 1&=_/E)?)*&:+#G&>2+'?&i&'*&=#$*Z/J)>=*#j&D+i&*&>=#$*

Documents

Remote Conformance & Interop Testing F-Interop · 2016. 10. 7. · 1 - TPAC2016 – WoTIG Meeting, 22nd September 2016 Remote Conformance & Interop Testing 1 F-Interop Online Platform