Automatic Testing of Neighbor Discovery Protocol Based on FSM and TTCN

Zhiliang Wang, Xia Yin, Haibin Wang, Jianping Wu

Department of Computer Science

Tsinghua University

http://netlab.cs.tsinghua.edu.cn

APCC’2004

Sep. 1, 2004

APCC'2004 2

Outline• Background

• Motivation

• Our proposed method

• Conclusion and Future work

APCC'2004 3

Background: Protocol Conformance Testing

Protocol Specification

Test Generation

Abstract Test Suite

Test Implementation

Implementation Process

Protocol Impl.IUT

Test Execution

Test Verdict

APCC'2004 4

Background: Neighbor Discovery Protocol

• One of the basic protocols in IPv6 protocol set

• Should be supported in all IPv6 implementations

• Corresponding functions in IPv4– ARP (Address Resolution Protocol)

– ICMP router discovery

– ICMP redirect function

• Using 5 ICMP packet types:– Router Solicitation

– Router Advertisement

– Neighbor Solicitation

– Neighbor Advertisement

– Redirect

APCC'2004 5

Background: Related work• Conformance testing of ND Protocol

– InterOperability Laboratory, University of New Hampshire

– TAHI Project

– Institute of Computing Technology, Chinese Academy of Sciences

• Existing Problems– No standard language used to specify test suite

– Must be executed on the proprietary test systems

– Not guided by formal methods

– Cannot ensure test coverage and reliability of test suite

APCC'2004 6

Background: Our method• FSM

– Finite State Machine

– To specify protocol specification

• TTCN-2– Tree and Tabular Notations

– Test case specification language

• PITS– Our previous work

– Protocol Integrated Test System

– TTCN-based test system

– Automatic test process

– Test practice

Protocol Specification

Test Generation

Abstract Test Suite

Test Implementation

Implementation Processes

Protocol Impl.IUT

Test Execution

Test Verdict

FSM

TTCN

PITS

APCC'2004 7

Outline• Background

• Motivation

• Our proposed method

• Conclusion and Future work

APCC'2004 8

Motivation• Protocol Modeling

– Control part

– Data part

– Timed part

• ND Protocol includes simple Timer Operations!

• How to specify it?– EFSM? (Extended FSM)

• Control part and data part, no timer!

– TIOA? (Timed Input Output Automata)• Common model for timed system

• Test generation process has tremendous costs on time and space

• Difficult to be applied in practical conformance testing

– Our method: EFSM + simple timer operations

APCC'2004 9

Outline• Background

• Motivation

• Our proposed method– Formal Model

– Test Architecture

– Test Generation

– Test Practice

– Test Results and Analysis

• Conclusion and Future work

APCC'2004 10

Formal Model• EFSM with Simple Timed Extensions

– EFSM + simple timer operations

– A local timer for each state

– Transition of such a model: • tact (timed action): invoke the local timer start(tq)

• Timeout transition: input is timeout(ts)

qs otactupopPi };;/{][

INCOMPLETE REACHABLE

NONE

t5

t3

t5: ?NA [NA.S=1]/{LLA:=NA.TLLA;start(TR)}!ERep

t3: timeout(TI) [counter=3]/{counter:=0}

i: Input P: Predicate

up: variable update function

i: Input P: Predicate up: variable update function

tact: timed action

o: output

APCC'2004 11

Formal Model (cont)Protocol specification of Neighbor Unreachability Detection in ND Protocol

INCOMPLETE

t1

PROBE

REACHABLE

STALE

DELAY

t2

t3

t4

t5

t6

t7

t8

t9

t10

t11

t12

t13t14t15

t16

t17

t18t19t20

t21

t22

t23

t24

t25 t26

t27t28

t29

t30

t31 t32

t33

t34

t35

t36

t37

t38

t39

t41

t40

t42

t43

t44t45

NONE

States: 6Transitions: 45

APCC'2004 12

Formal Model (cont)Protocol specification of Neighbor Unreachability Detection in ND Protocol

Trans Protocol Behavior Description Trans Protocol Behavior Description

t1 ?EReq/{counter:=1;start(TI)}!mNS t13,t25,t31 ?NA[NA.S=0;NA.O=0]/--

t2 timeout(TI)[counter<3]/{counter:=counter+1;start(TI)}!mNS t19 ?NA[NA.S=0;NA.O=0]/{start(TR)}

t3 timeout(TI)[counter=3]/{counter:=0} t14,t26,t32 ?NA[NA.S=0;NA.O=1;NA.TLLA=LLA]/--

t4 ?NA[NA.S=0]/{LLA:=NA.TLLA}!ERep t20 ?NA[NA.S=0;NA.O=1;NA.TLLA=LLA]/{start(TR)}

t5 ?NA[NA.S=1]/{LLA:=NA.TLLA;start(TR)}!ERep t15,t21,t27,t33 ?NA[NA.S=0;NA.O=1;NA.TLLA!=LLA]/{LLA:=NA.TLLA}

t6 ?EReq/{start(TD)}!ERep t34 ?NA/--

t7 timeout(TD)/{counter:=1;start(TP)}!uNS t35 timeout(TR)/--

t8 timeout(TP)[counter<3]/{counter:=counter+1;start(TP)}!uNS t36 ?OTHER/{LLA:=OTHER.SLLA}

t9 timeout(TP)[counter=3]/{counter:=0} t37 ?OTHER/{LLA:=OTHER.SLLA}!ERep

t10,t16,t22,t28 ?NA[NA.S=1;NA.O=0;NA.TLLA=LLA]/{start(TR)} t38,t39,t40,t41 ?OTHER[OTHER.SLLA!=LLA]/{LLA:=OTHER.SLLA}

t11,t17,t23,t29 ?NA[NA.S=1;NA.O=0;NA.TLLA!=LLA]/-- t42,t43,t44,t45 ?OTHER[OTHER.SLLA=LLA]/--

t12,t18,t24,t30 ?NA[NA.S=1;NA.O=1]/{LLA:=NA.TLLA;start(TR)}

APCC'2004 13

Formal Model (cont)• External Observable timed features

– Do not apply any external inputs to the machine in state s. If an external output o can be observed after waiting a while, state s has an External Observable Timed Feature and its observed output is o.

– the external behaviors of timeout transitions

– To verify the transient state

• Example

State Has such a

feature?

Observed

output

Wait

time(s)

NONE N -- --

INCOMPLETE Y mNS (0,1)

STALE N -- --

REACHABLE N -- --

DELAY Y uNS (0,5)

PROBE Y uNS (0,1)

APCC'2004 14

Test Architecture

IUT timed part

Test System timer

PCO

PCO: Point of Control and ObservationIUT: Implementation Under Test

APCC'2004 15

Test Generation• Conformance test suite

– State cover: for each state S

– Transition cover: for each transition t

S0 S?r pre(S) ds(S)

pre(S): Preamble

ds(S): Distinguishing Sequence

? State cover test case

S0 S?r pre(S) ds(Q)

?Qt

Transition cover test case

Lead the machine to the initial state S0

Lead the machine from S0 to state S

Transition to be tested

Verify the final state is S

Verify the final state is Q

r: Reset

State to be tested

APCC'2004 16

Test Generation (cont)• Generation of Distinguishing Sequence

– Extension of traditional method• Key idea: Consider Timed Features of model

{N,I,S,R,D,P}Wait

mNS(1s)

uNS(1s)

uNS(5s)

--

I P D {N,S,R}EReq

mNS ERep

N {S,R}Wait

uNS(5s)

--

S R

Example:

Distinguishing Seq. of state S:

Wait/--; ?EReq/!EPep; Wait(5s)/!uNS

APCC'2004 17

Test Generation (cont)• TTCN-2 Test case example:

Test Case Dynamic Behavior

Test Case Name：P_Recv_SNA_O0_SameLLA_R

Group： NUD_FSM/ TRANS_COVER

Purpose： When NUD state is PROBE, IUT receives an SNA (with O=0, same LLA), IUT's NC entry

will be lead to REACHABLE (transition t28).

No. Behavior Description Constraint. Ref Verdict Comments

1 +To_PROBE (1)

2 PCO1!NA_TLLA_t SNA_O0_SameLLA (2)

3 +Check_REACHABLE (3)

Detailed Comments:

(1) Test step1: Preamble to lead the IUT from init state to PROBE;

(2) SNA_O0_SameLLA is an NA with S=1, O=0, same LLA (input of t28);

(3) Test step2: Distinguishing sequence for state REACHABLE: Verify the final state of this transition is REACHABLE

APCC'2004 18

Test Generation (cont)Test Step Dynamic Behavior

Test Step Name：Check_REACHABLE

Purpose： Check if IUT is in state REACHABLE.

No. Behavior Description Constraint. Ref Verdict Comments

1 START WaitTimer_10 (1)

2 PCO1?OTHERWISE FAIL (2)

3 ?TIMEOUT WaitTimer_10

4 PCO1!EReq_t ICMP6_EReq Input a echo request to IUT

5 START WaitTimer_3 (3)

6 PCO1?ERep_t ICMP6_ERep IUT should respond a echo reply

7 START WaitTimer_10 (1)

8 PCO1?OTHERWISE FAIL (2)

9 ?TIMEOUT WaitTimer_10 PASS

10 PCO1?OTHERWISE FAIL

11 ?TIMEOUT WaitTimer_3 FAIL

Detailed Comments:

(1) Wait timer to check External Observable timed features should be longer than 5s.

(2) If the current state has an External Observable timed feature, then FAIL (according to ADT).

(3) A guard clock to protect test system from deadlock, used before any receive events.

APCC'2004 19

Test Practice• Test System

PITSIUTNDP TTCN

Test Suite

IPv6 link

PCO1 Hub1

Test Execution

RI

timed part

NDP Module

timer

RI: Reference ImplementationPCO: Point of Control and ObservationIUT: Implementation Under Test

APCC'2004 20

Test Practice (cont)• RI (Reference Implementation)

RI

Main Thread

Test Execution

Massage Processing

Packet Sending

Packet Receiving

Thread

IUTTelnet

Automatic Configuration

Lower Layer Protocol Services

Test System

APCC'2004 21

Test Results and Analysis• Overview of NDP test suite

• Test Result

Test Group Test Purpose Test Cases Num.

Basic Basic Function of Router Discovery 9 Router_Discovery

NUD_Recv_RS Test NUD FSM: when receiving an RS 10

Address_Resolution Basic Function of Address Resolution 22

STATE_COVER Test NUD FSM: state cover 6

TRANS_COVER Test NUD FSM: transition cover 36 NUD_FSM NUD_Recv_NS Test NUD FSM: when receiving an NS 8

Total Number 91

Implementation Pass Number Pass Rate(%)

Product A 85 93.4

Product B 78 85.7

Prototype I 66 72.5

Prototype II 89 97.8

Statistics and comparison of pass rate

93.485.7

72.5

97.8

0.0

20.0

40.0

60.0

80.0

100.0

routerimplementations

% Product A

Product B

Prototype I

Prototype II

APCC'2004 22

Conclusion and Future work• A formal method to test neighbor discovery protocol

– Protocol specification: FSM-based method

– Test generation

– Test specification: TTCN-2

– Test practice: PITS

• Future work:– Data flow testing for such a model

– further test activities on IPv6 protocols, especially routing protocols

Thank you!

Q&A