ESVT: A Toolkit Facilitating Use of DETER

Lunquan Li, Jiwu Jing, Peng Liu, TJ, Jisheng, George

Kesidis, David Miller

Penn State UniversitySeptember 28, 2005Newport Beach, CA

2

Motivation• Specific testbeds need specific tools

– EMIST tools are DETER specific• Tools are a vehicle to make the evaluation methods

developed by EMIST available to experimenters• EMIST tools make DETER experiments easier• EMIST tools save the experimenters’ time and energy

Experimenter

EMIST tools

General purpose tools

DETER

3

EMIST Tool Effort PSU ESVT toolkit UCD NTGC network traffic generation and control tool ICSI/PSU worm scale-down equations UCD emulated worm attack generation tool PSU KMSim Slammer-like attack generator SRI/UCD worm simulation tools UCD XML worm specification tool UCD BGP routing data viz tool PSU NTD traffic data mining tool Purdue scriptable event system Purdue sys info logging tool SPARTA/McAfee DDOS trace analysis and viz scripts Purdue data analysis and viz scripts

4

ESVT: Status ESVT 1.0 -- May 2004

Windows platformC++User manualSample DETER experiment package

ESVT 2.0 -- May 200534,494 lines of C++ code

ESVT made open source in July 2005

Download http://emist.ist.psu.edu

ESVT 1.0 Executable: 70 times

ESVT 2.0 Executable: 26 times

ESVT 2.0 Source code: 12 times

Downloads:

5

EMIST Tool Design SpacePre-Execution Post-ExecutionExecution

-- Draw topology-- Import topology-- Configure a node-- Setup virtualization-- Generate TCL scripts-- Setup meters-- Upload programs-- Setup trace logger-- Configure bandwidth, latency, etc. -- Specify attacks-- etc.

-- Attack injectors-- Background traffic generators-- Replay trace data-- Trace logger-- Event logger-- Meters-- Virtual nodes-- Internet interface simulator-- Event coordination-- Conf. tracking-- Pause, reconfigure, resume -- etc.

-- Trace analysis (scripts)-- Visualization-- Traffic data mining-- Data aggregation-- Animation, replay-- Database integration-- User-defined views-- TCPDUMP2Netflow-- Analysis workflow learning-- etc.

6

ESVT Overview

Pre-Execution Post-ExecutionExecution

-- Draw topology

-- Import topology

-- Configure a node

-- Setup virtualization

-- Generate TCL scripts

-- Configure bandwidth,

latency, etc.

-- Specify attacks

-- Attack packet

injectors* (KMSim)

-- Trace logger*

-- Virtual nodes*

-- Internet interface

simulator*

-- Visualization

-- Traffic data mining*

-- Data aggregation

-- Animation, replay

-- Database integration

-- User-defined views

-- TCPDUMP2Netflow

* To be integrated.

-- May 2004: Version 1.0 -- May 2005: Version 2.0

Step 1. Setup the experiment using ESVT

Step 2. Setup the DETER environment

Step 3. Run the experiment on DETER

Step 4. Visualize the results using ESVT

- EMIST topology specification in TCL - Virtual sub-network nodes - Internet interface - Normal & vulnerable nodes - Bandwidth, latency, addresses, OS- Other auxiliary TCL scripts

- Worm program- Traffic generator program- Internet interface program- Virtual node program- Normal node program- Vulnerable node program- TCPDUMP setup- EMULAB GUI can be used here

- Worm propagation snapshots- Worm propagation animation- Link traffic bar chart (dynamic)- Worm replay

8

Year 3 Themes of ESVT• BGP ESVT• Integration

– Integrate ESVT into the broader SEW (Security Experimenter’s Workbench) concept

– Integrate NTD and other trace audit tools into ESVT

• Support PREDIT– Use ESVT to help experimenters understand the

characteristics of various DHS data sets

9

ESVT Screenshots

Demo: this afternoon

10The topology of the worm experiment done by Nick Weaver et al. in 2004.

11Enterprise topology: 925 hosts, 70 switches, 7 routers

router

InternetInterface

Host

Switch

12A topology imported from GT-ITM format.

13Node configuration in a zoomed-in topology.

14A TCL script generated by ESVT: support virtualization; set up trace loggers; set up the Internet interface; etc.

set lan70 [$ns make-lan "$n(969) $n(978) " 100Mb 0ms]#--Total Switch: 3, Computer: 58, Susceptible ones: 1.set link969 [$ns duplex-link $n(979) $n(977) 100Mb 0ms DropTail]# Running programs sectiontb-set-node-startcmd $n(902) "/proj/worm/e1k/scripts/run_virtual n-902-lan3 160"tb-set-node-startcmd $n(903) "/proj/worm/e1k/scripts/run_virtual n-903-lan4 160"tb-set-node-startcmd $n(936) "/proj/worm/e1k/scripts/run_virtual n-936-lan37 160“……..tb-set-node-startcmd $n(943) "/proj/worm/e1k/scripts/run_virtual n-943-lan44 160"tb-set-node-startcmd $n(945) "/proj/worm/e1k/scripts/run_tcp 945 160"tb-set-node-startcmd $n(946) "/proj/worm/e1k/scripts/run_virtual n-946-lan47 160"tb-set-node-startcmd $n(969) "/proj/worm/e1k/scripts/run_virtual n-969-lan70 160"tb-set-node-startcmd $n(972) "/proj/worm/e1k/scripts/run_tcp 972 160"tb-set-node-startcmd $n(973) "/proj/worm/e1k/scripts/run_tcp 973 160"tb-set-node-startcmd $n(974) "/proj/worm/e1k/scripts/run_tcp 974 160“……tb-set-node-startcmd $n(978) "/proj/worm/e1k/scripts/run_tcp 978 160"tb-set-node-startcmd $n(979) "/proj/worm/e1k/scripts/run_internet 979 160"$ns rtproto Static$ns run

#network address/prefix10.1.1.1/16#node & virtual node map file#n-#### TYPE(B/I/V/R) S/N #####(GUI node index) #####(Last segment of IP)n-902 V N 29 254n-902 V N 27 253n-902 V N 32 252n-902 V N 36 251n-902 V N 38 250n-902 V N 40 249n-902 V N 43 248

15

-- Use a SQL query to instrument a network-wide traffic view. -- MySQL database integration.-- Support both TCPDUMP and NetFlow formats.

16Data sources for link visualization are defined by a SQL query

17User-defined link visualization: options to define views

18Sample visualization output. Click on any plot will zoom-in and show further details.

19

Animation: the network event replay toolbar with a pop-up link traffic chart.

20BGP ESVT – the first shot.

21

Questions?

22

PSU KMSim Slammer-like Attack Generator

• KMSim is a simulation code, consisting of coupled Kermack-McKendrick epidemic equations, to model the spread of a bandwidth-limited, randomly scanning Internet worm

• Benefit: a family of worms can be flexibly simulated by tuning few parameters

23

PSU NTD Traffic Data Mining Tool

• This tool can detect the significant clusters, i.e., clusters whose traffic is greater than a threshold (either in terms of packet number or bytes) – Cluster definition: source IP, destination IP, source

port, destination port or protocol• NTD is an efficient implementation of that

described by Estan et al. in SIGCOMM ’03• NTD is offline• A tool for efficient mining of the multidimensional

traffic cluster hierarchy for digesting, visualization, and modeling

24

EMIST Tool Effort ICSI/PSU worm scale-down equations PSU ESVT toolkit* PSU KMSim Slammer-like attack generator* PSU NTD traffic data mining tool* Purdue scriptable event system* Purdue sys info logging tool* Purdue data analysis and viz scripts* SPARTA/McAfee DDOS trace analysis and viz scripts SRI/UCD worm simulation tools UCD emulated worm attack generation tool UCD NTGC network traffic generation and control tool UCD XML worm specification tool UCD BGP routing data viz tool * Officially released

25

Purdue Scriptable Event System

• During a DETER experiment, many events may happen– time events, cmd events, etc.

• Although local event response can be pre-programmed on a single test machine, synchronized event response among a set of test machines cannot be pre-programmed

• This tool allows runtime coordinated event response via a coordinator-participant model

• Each test machine can run a participant stub that communicates with the coordinator to report events and receive response instructions

• The global event response plan can be flexibly scripted by the experimenter

26

Purdue Sys Info Logging Tool• This tool logs system level statistics associated

with a certain network interface timestamp, bytes_per_sec, pack_per_sec, bytes_per_sec_up,pack_per_sec_up, memtotal, memused, uptime, idletime, established TCP connections, half open TCP connections,TCPSlowStartRetrans count,TCPAbortOnTimeout count,errs on the device drivers, drops on the device drivers

27

UCD Emulated Worm Attack Generation

•All nodes host a worm generation daemon.

•Nodes wait for worm attack “instructions”.

•Propagation behavior of worm is varied by varying the “instructions”.

•An XML specification of worm propagation serves as the instructions.

28

UCD Network Traffic Generation and Control (NTGC)

Raw trace 1

Raw trace n

……

……

……

…

Traffic Analyzer

Reconstruct TCP connections

Generate flow dataMerge tracesTimestamp normalization

Connection Data

Flow Data

Traffic Filter

Filtering

Address RemappingScale up/ downDuplicateRemove

Address Remapping rules.Topology file

Configuration File Generator