Automatic test packet generation

Automatic Test Packet

Generation

Tushar Jadhav

Contents

Introduction

Problem Statement

Concept

Conclusion

References

Introduction

Outages in world

Network Troubleshooting

Downtime:

25%take over an hour to resolve. (estimated $60K-110K/hour ).

Current tools:

Ping, Traceroute, SNMP.

70%asked for better tools, automatic tests

Automatic Test Packet Generation

(ATPG) generates packets :

- To test the liveness of the underlying

topology .

- To test the compatibility between

data plane state and configuration

specifications.

- To test performance assertions such

as packet latency.

Problem Definition

Most common causes of network

failure

1) Hardware failures

2) Software bugs

Others failure’s include reachability

failures and throughput/latency

degradation.

Our goal is to automatically detect

these types of failures.

Goals

Automatically generate test packets to

test the network state, faults before

being

noticed by application.

Augment human wisdom and intuition.

Reduce the downtime.

Save money.

Concept

ATPG System :

Roadmap

Reachability Analysis

Test Packet Generation & Selection

Fault Localization

Implementation & Evaluation

Reachability Analysis

Header Space Analysis

All-pairs reachability : Compute all

classes of packets that can flow

between every pair of ports.

All-pairs reachability

Test Packet Generation &

Selection

All-pairs reachability

Test Packet Selection

Packets in all-pairs reachability table

are more than necessary.

Goal :

Select a minimum subset of packets

whose histories cover the whole rule

set.

Min Set Cover

Test Packet Selection

Min Set Cover

-Exercise all rules

-Sent out

Periodically

- “Redundant”

- Will be used in

Fault Localization

Fault Localization

Fault Localization

In ATPG: we can choose packets!

Step 1:

Use results from regular test packets– F (potentially broken rules) = Union from all failing packets

– P (known good rules) = Union from all passing packets

– Suspect Set = F –P

Step 2 :

Use reserved test packets

- Pick packets that test only one rule in the suspect set, and send them out for testing

- Passed: eliminate

- Failed: label it as “broken”

Step 3:

Brute force

- Continue with test packets that test two or more rules in the suspect set, until the set is small enough

Implementation

Putting Them all together :

Cisco/Juniper Parsers

-Translate router configuration files and forwarding tables (FIB) into Header space representation.

Test Packet Generation/Selection

- Hassel: A python header space library

- Min-Set-Cover

- Python’s multiprocess module to parallelize

SDN can simplify the design

ATPG for Performance

Testing

ATPG can also be used for detecting and localizing performance problems.

Intuition: generalize results of a test from success/failure to performance (e.g. latency)

Users can inject performance errors

Limitations

Dynamic/Non-deterministic boxes

- e.g. NAT

“Invisible” rules

- e.g. backup rules

Transient network states

Ambiguous states (work in progress)

- e.g. ECMP

Conclusion

Test’s the forwarding state by

generating minimal link, queue, rule

cover automatically.

Efficient liveness testing,reachability

testing.

Better tool than ping & traceroute.

Brings more testing and coverage of

networks

References

P. Kazemian, G. Varghese, and N. McKeown, “Header space analysis : Static checking for networks,” in Proc. NSDI, 2012, pp. 9–9.

Y. Bejerano and R. Rastogi, “Robust monitoring of link delays and faults in IP networks,” IEEE/ACM Trans. Netw., vol. 14, no. 5, pp.1092–1103, Oct. 2006

A. Dhamdhere, R. Teixeira, C. Dovrolis, and C. Diot, “Netdiagnoser:Troubleshooting network unreachabilities using end-to-end probes and routing data,” in Proc. ACM CoNEXT, 2007, pp. 18:1–18:12.