Upload
galvin
View
25
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Verisim: Formal Analysis of Network Simulations. Karthikeyan Bhargavan, Carl A. Gunter , Moonjoo Kim, Insup Lee, Davor Obradovic, Oleg Sokolsky, Mahesh Viswanathan University of Pennsylvania. Internetwork Routing Protocols. - PowerPoint PPT Presentation
Citation preview
Verisim: Formal Analysis of Network Simulations
Karthikeyan Bhargavan, Carl A. Gunter, Moonjoo Kim, Insup Lee, Davor Obradovic, Oleg Sokolsky, Mahesh Viswanathan
University of Pennsylvania
Internetwork Routing Protocols Internetwork routing protocols
enable interoperability between physical networks.
Routing protocols for the Internet: RIP, OSPF, and BGP.
Routing protocols for packet radio: AODV and DSR.
Routing protocols and software have growing importance and complexity.
Current Analysis Techniques
Rigorous mathematical models and proofs – limited by complexity.
Testing. Testbed – expensive. Operational – risky and inconvenient.
Simulation. Performance attributes: throughput,
latency, reliability, etc.
Limitations of Performance Analysis of Simulations
Flaws may not be detected if no expected performance can be used for comparison.
When flaws are suspected, finer means of analysis are useful.
Some flaws do not manifest themselves as performance problems (e.g. most security gaps).
Verisim
Verisim provides support for “logical” analysis of network simulations.
This talk describes its architecture and logic.
We provide a series of experiments aimed at assessing the approach.
Monitoring and Checking (MaC) Framework
MonitoringScript
Program
Monitor Checker
Properties
MetatraceTrace
Data
NS MEDL
Meta Event Description Language (MEDL)
Expresses properties of traces. Extension of Linear Temporal Logic
(LTL) with auxiliary variables. More expressive than LTL. Properties classified into
(instantaneous) events and (enduring) conditions.
S Kannan, M Kim, I Lee, O Sokolsky, M Viswanathan 98
NS Network SimulatorInstrumentedProtocol Code
P : C++
ConfigurationParameters:
OTcl
TrafficModel:OTcl
Topology:OTcl
Scenario
P
N
src/sink
P
N
src/sink
P
N
src/sink
ProtocolAgents
NetworkModel
TrafficAgents
NS TraceVINT Project
Verisim
Trace Checker
Properties: MEDL
Metatrace
InstrumentedProtocol: C++
Scenario:OTcl
NS
Ad Hoc Networks
Routing for a wireless internetwork without the aid of a central base station.
Connections are low-bandwidth, lossy, and highly transient.
Unique routing assumptions: Most routes are seldom used. Bandwidth must be protected.
Illustration Part 1 of 2
Routing
Movement
Illustration Part 2 of 2
NewRouting
AODV Protocol If a node S needs a route to a destination D
and does not have one, it floods a route-request (RREQ) packet through the network.
Each recipient R of this RREQ keeps a return pointer.
R broadcasts the request to its neighbors if it is not D and does not have a route to D.
If R is D, or has a route to D, it responds with a route-reply (RREP) packet using the return pointers for S.
Perkins and Royer 99
Loop Freedom in AODV Routing loops are undesirable. AODV uses sequence numbers to
indicate freshness of link information.
Key Invariant: If next(n) = n’, then seqno(n) seqno(n’), and if seqno(n) = seqno(n’), then hops(n) >
hops(n’). The invariant ensures that there are
no loops.
Outline of Experiment Run a scenario of modest complexity. Analyze it in Verisim using a list of 9
properties of AODV expressed in MEDL. First pass: Repair First Bug (RFB). Second pass: “tune” the MEDL formulas to
avoid rerunning the simulation discovering bugs in the metatrace.
We instrumented simulation code for AODVv0 supplied by the CMU Monarch Project.
Experiment Scenario1
2 3
4
5
Experiment Scenario1
2 3
4
5
Experiment Scenario1
2 3
4
5
Experiment Scenario1
2 3
4
5
Experiment Scenario1
2 3
4
5
Sample MEDL Alarmalarm LoopInv[at][nxt][dst] = sendrout[at][dst] when ((atnxt) (atdst) (nxtdst) (obs_nexthop[at][dst] == nxt) ((obs_seqno[at][dst] > obs_seqno[nxt][dst]) ((obs_seqno[at][dst] == obs_seqno[nxt][dst]) (obs_hopcont[at][dst] <= obs_hopcnt[at][dst]))))
This is the negation of the fundamental invariant ensuringno loops in AODV:
If the next hop from node at toward destination dst is node nxt then the sequence number (for dst) of at is less than or equal to that of nxt or they are equal and the hop count (for dest) of at is less than or equal to that of nxt.
Repair First Bug Experiment Let be the MEDL property set. Run the simulation to get a trace T. Run the checker to get a metatrace T. Repair the first bug in the metatrace to
get new protocol code. Rerun the simulation with the new
protocol code to get a new trace U. Rerun the checker to get a new metatrace
U. Continue until an empty metatrace is
obtained.
RFB Experiment Statistics
Meta-trace
DestRep
DetectRErr
NodeRep
LoopEnv
TotalAlarms
T 4 54 38 43 220
U 0 54 38 43 216
V 0 48 39 44 206
W 0 0 0 0 1
Tuning Experiment Statistics
Meta-trace
DestRep
DetectRErr
NodeRep
LoopEnv
TotalAlarms
T 4 54 38 43 220
T 0 54 38 43 216
T 0 0 38 50 166
T 0 0 21 0 21
No newsimulation
traces.
Analysis of “Off-The-Shelf” (OTS) Simulations The prior experiment involved fewer
than 10,000 events, and we designed it to exercise key scenarios.
Can we do useful analysis with OTS performance simulations?
Aim: Verisim can be added with small modifications, run alongside OTS simulations, and find logical failures in a practical period of time.
Experiment Took largest available OTS simulation for
AODV and ran it to create a trace. Simulation from Monarch uses 50 mobile nodes on
1500x300m grid moving 20 m/s. 5220 seconds (1.5 hours) to complete simulation. 6,446,316 events.
Naïve effort: ran MEDL with MonSeqNo test on all nodes (2500 relations) using 550Mhz dual processor machine with 1GB of memory.
Aborted the experiment after 4 days; estimate 100 days to complete this analysis.
Optimizations For Network Routing Simulations
Population Abstraction: test the property for a subset of the routers.
Packet Type Abstraction: prune the trace to include only relevant events.
Sample Experiment Results for MonSeqNo Population Abstraction for 5 nodes.
Trace size: 6,446,316 events Property size: 14,543 bytes Time: 51,054 seconds Rate: 0.54 micro seconds per event per property.
Population and Packet Type Abstractions. Trace size: 6812 events Property size: 14,543 bytes Time: 51 seconds (found failures) Rate: 0.51 micro seconds per event per property.
Sample Experiment Results for LoopInv Population Abstraction for 5 nodes.
Trace size: 69,411 events Property size: 75,508 bytes Time: 8064 seconds Rate: 1.54 micro seconds per event per property.
Population and Packet Type Abstractions. Trace size: 48,735 events Property size: 75,508 bytes Time: 5912 seconds (found failures) Rate: 1.61 micro seconds per event per property.
Related Work Communication-based test generation
systems. STRESS and Verisoft. Test oracles based on formulas or
formal operational specifications. GIL and TETRA.
Formal instance verification of routing protocols. SPIN/PITHIA for PNNI.
Simulations based on logical specifications. MTSim and Maude.
Network monitoring systems. Emerald.
Summary Verisim integrates simulation and
logical trace analysis. The combination provides a more
flexible approach to analyzing network simulations for safety properties.
It is able to find bugs in existing simulator studies.
Its flexibility can be exploited to improve debugging turn-around time.