Upload
aviv
View
41
Download
0
Embed Size (px)
DESCRIPTION
DETER/EMIST DDoS Defense Experimental Methodology Impact of Traffic Generation Selection on Precision of Detection and Response Characterization. Stephen Schwab September 28, 2005. SPARTA Team Participants. DETER Steve Schwab, Ron Ostrenga, Brad Harris, David Balenson, Dan Sterne EMIST DDoS - PowerPoint PPT Presentation
Citation preview
Schwab_20050928_1
DETER/EMISTDETER/EMISTDDoS Defense Experimental MethodologyDDoS Defense Experimental Methodology
Impact of Traffic Generation Selection on Precision of Detection and Impact of Traffic Generation Selection on Precision of Detection and
Response CharacterizationResponse Characterization
Stephen SchwabSeptember 28, 2005
Schwab_20050928_2
SPARTA Team ParticipantsSPARTA Team Participants
• DETER– Steve Schwab, Ron Ostrenga, Brad Harris, David
Balenson, Dan Sterne• EMIST DDoS
– Steve Schwab, Brett Wilson, Ron Ostrenga, Roshan Thomas, Alefiya Hussain, Brad Harris, Dan Sterne
Schwab_20050928_3
OverviewOverview
• EMIST DDoS Experimental Methodology
• FloodWatch and CloudShield Experiment Design
• Testbed Methodology Automation
• Process for Incorporating Hardware and Appliances
• Traffic Generation
• Visualization and Metrics
• Future Plans
Schwab_20050928_4
Objectives for EMISTObjectives for EMIST
• Create reusable library of test technology for conducting realistic, rigorous, reproducible, impartial tests
– For assessing attack impact and defense effectiveness– Test data, test configurations, analysis software, and experiment
automation tools• Provide usage examples and methodological guidance
– Recommendations for selecting (or developing) tests and interpreting results
– Test cases and results, possibly including benchmarks• Facilitate testing of prototypes during development and commercial
products during evaluation
Schwab_20050928_5
Canonical DDoS ExperimentCanonical DDoS Experiment
• DDoS experimentation involves a vast multidimensional space of experimental variables.
• Canonical experiment form intended to organize the experimental space and facilitate navigation through it.
• Canonical experiment consists of:1. Attack Mechanisms2. Background Traffic3. Network Topology4. Defense Mechanisms5. Measurements and Metrics6. Network Services Infrastructure7. Risk
Schwab_20050928_6
Typical DDoS Toolkit ArchitectureTypical DDoS Toolkit Architecture
Hacker Console Master
Zombie
Victim
Schwab_20050928_7
OverviewOverview
• EMIST DDoS Experimental Methodology
• FloodWatch and CloudShield Experiment Design
• Testbed Methodology Automation
• Process for Incorporating Hardware and Appliances
• Traffic Generation
• Visualization and Metrics
• Future Plans
Schwab_20050928_8
SPARTA DDoS ExperimentSPARTA DDoS ExperimentSeptember 2005September 2005
Background Traffic:REPLAY | NTCG | HARPOON
HIGH FIDELITY TRAFFIC
Topology:BUILDING-BLOCKS |
JUNIPER ROUTER CORE
REALISTIC CONNECTIVITY AND SCALE-DOWN
Attack Traffic:DETER-INTEGRATED ATTACK SCRIPTING
AUTOMATION OF VARIETY OF SCENARIOS UNDER STUDY
Instrumentation:PACKET AND HOST STATISTICS CAPTURE | SPECTRAL ANALYSIS | METRICS CALCULATION | INTEGRATED VISUALIZATION
TOOLBOX FOR RIGOROUS INVESTIGATION OF RESULTS
COREAS-11357
ATTACK TRAFFICBACKGROUNDTRAFFIC
Schwab_20050928_9
SPARTA DDoS ExperimentSPARTA DDoS ExperimentSeptember 2005September 2005
Defenses:
FloodWatchSTATISTICAL DDOS DETECTION
McAfee IntruShieldCOMMERCIAL IDP APPLIANCE
CloudShieldNETWORK PROCESSOR APPLIANCE PLATFORM
Juniper IDP-200COMMERCIAL IDP APPLIANCE
STUDY AND QUANTIFY HOW PLACEMENT WITHIN THE NETWORK IMPACTS EFFECTIVENESS OF DEFENSE
DEFENSEDEPLOYMENTPOINT
Schwab_20050928_10
FloodWatch Experiment – Example TopologyFloodWatch Experiment – Example Topology
Each end network (leaf and root) is both a source and sink for background traffic
Example traffic flow
From one edge network
Core Network
EdgeNetwork
Flood Watch
Root and leaf networks transmit/receive replayed packets
Packet source and destination addresses randomly remapped to experiment topology
Schwab_20050928_11
Schematic of Network ConnectivitySchematic of Network Connectivity
2nd TierRouter
CoreRouter
ApplianceCore
Router2nd TierRouter
HostHost
FloodWatch DDoSStatistical Detector
Juniper M7i
CloudShield 2200DDoS EntropyDetector
Target orAttack Victim
Attack SourceReplay Traffic SourceTCP Traffic Source
Schwab_20050928_12
Experiment GoalExperiment Goal
• Validate fidelity of attack and background traffic in reproducing characteristics of real DDoS experimental scenarios.
• Case #1: Attack Traffic– Recreate an attack captured in a real-world network– Compare spectral analysis of real-world and testbed network
to assess fidelity of phenomena reproduction.
• Case #2: Background Traffic– Use TCP analytical model of throughput as a reference– Compare theoretical throughput with observed throughput for
100Mb/s and 1Gb/s networks– Compare model-vs-testbed discrepancy between100Mb/s and
1Gb/s to gauge ability to preserve phenomena while scaling-up traffic.
Schwab_20050928_13
OverviewOverview
• EMIST DDoS Experimental Methodology
• FloodWatch and CloudShield Experiment Design
• Testbed Methodology Automation
• Process for Incorporating Hardware and Appliances
• Traffic Generation
• Visualization and Metrics
• Future Plans
Schwab_20050928_14
AutomationAutomation
• DDoS Requirements generalize to broader Security Experiment Requirements:
• Automate whenever possible for:– Repeatability, efficiency, ease-of-use
• Experiments described in Emulab ns2 format may include primitive events
– Under base Emulab system, provides control for a very limited number of operations
Schwab_20050928_15
Event System IntegrationEvent System Integration
• Testbed Automation Agents are controlled by events• Events can be created in several ways:
– Specified in the NS file and played at runtime– Played from an external list of events generated by
hand or script– Dynamically generated by a GUI or script
• Current Agents– Injector: traffic replay agent– Flooder: attack agent– Collector: packet traces (TCPDUMP) control and per-
node filter counters– Harpoon: traffic generation agent for the Harpoon TG– FloodWatch: controls FloodWatch DDoS defense
Schwab_20050928_16
Automation DashboardAutomation Dashboard
Schwab_20050928_17
OverviewOverview
• EMIST DDoS Experimental Methodology
• FloodWatch and CloudShield Experiment Design
• Testbed Methodology Automation
• Process for Incorporating Hardware and Appliances
• Traffic Generation
• Visualization and Metrics
• Future Plans
Schwab_20050928_18
AppliancesAppliances
• DETER requirement: support the experimental test and evaluation of appliances
– Commercial products often packaged as appliances, critical future user segment
• EMIST requirement: high-speed appliances stress the testbed and the tools supporting our methodology
– Requirements:– Provide the ability to seamlessly integrate appliances
as nodes in testbed experiments– Stress all aspects of our methodology at line-rate
» Topology – Gigabit forwarding routers (Juniper)» Traffic – Greater aggregation» Data Capture – vanilla TCPDUMP inadequate
Schwab_20050928_19
DDoS Network Appliance Defense DDoS Network Appliance Defense Evaluation ScenariosEvaluation Scenarios
• Introduce scenarios in which technology developers evaluate network appliances
– CloudShield 2200» IXP2850 network processor + high-level application
development environment» Evaluate prototype statistical DDoS defenses
– McAfee IntruShield 2600» Commercial product, focus on DDoS capabilities
– Juniper IDP-200» Commercial product with 8 Gigabit ports enabling study
of placement and connectivity– Push envelope of DDoS defense evaluation methodology to
test Gigabit Ethernet rates
Schwab_20050928_20
DETER Hardware and AppliancesDETER Hardware and AppliancesProcessProcess
• Develop a systematic process for integrating hardware devices and appliances in the DETER testbed and within EMIST experiments:
– Hardware connection– Control plane ns topology– Control plane manual configuration– Data plane manual configuration– Control and Data plane semi-automatic configuration
(scripting)– Control and Data plane automation
» Integrate generalized scripts behind the scenes into DETER and EMIST tools
Schwab_20050928_21
Juniper RoutersJuniper Routers
• Deter has 5 Juniper M7i routers• 4 Gigabit Ports/Router
• The Juniper routers in DETER are almost first-class DETER experimental devices.• Can be allocated into an experiment by Assign• Can be assigned IP addresses within ns topology
• Assign cannot YET configure router to use the IP addresses that were allocated
• Must manually map the MAC and IP addresses into a router configuration
• Plan to use the Juniper supported XML API to automatically configure the router
Schwab_20050928_22
CloudShield ApplianceCloudShield Appliance
• A CloudShield Appliance with 4 Gigabit interfaces has been added to DETER as an experimental device.
• Can be allocated into an experiment by Assign• Must be configured manually • Mapping of interfaces into an experiment is difficult
since there are no exposed MAC or IP addresses• Usage is complicated by the transparent bridging
function that causes the DETER switches to go into layer 2 loops.
• Spanning Tree Protocol (STP) is disabled on DETER
Schwab_20050928_23
CloudShield CA2200CloudShield CA2200
• 2 Intel IXP2800 network processors running CPOS– 16 microengines each– 1 StrongARM processor each
• Dual Pentium-III management processor running Red Hat Linux
• 4 GigE or 4 Copper 10/100/1000 network interfaces
Schwab_20050928_24
Entropy Based DetectionEntropy Based Detection
• Runs on CloudShield CA2200 network appliance• Data plane
– Detectors: source IP, destination IP, packet size, source port, destination port
– Create a histogram for each detector on packets in a fixed window size
– Calculate entropy for each detector– Optionally filter packets
• Control plane– Every N seconds, read entropy values– Compare with high/low thresholds for normal operation– When threshold is crossed, create a packet filter using
max item from each detectors histogram
Schwab_20050928_25
Entropy Detection IssuesEntropy Detection Issues
• Unable to run completely in the data plane– Packet processing runs in parallel– Entropy algorithm requires synchronization to avoid
race conditions– CPOS (CloudShield Packet OS) provides no support
for “manager threads” or mutexes
Schwab_20050928_26
Chi-Square AlgorithmChi-Square Algorithm
• The top bin contains the item with the highest count• Next bin contains the four next highest counts, etc.• Items are swapped to different bins as their relative
counts change• Chi-square statistic is the “shape” of the height of the
buckets
4 816
other
1
Schwab_20050928_27
Problems with Chi-SquareProblems with Chi-Square
• When a packet is received on the data plane, the item counter is incremented
• If the new count causes the item to be moved to another bin (because it has a higher count than one of the other items), the bins must be rebalanced
• Since packets are received in parallel, it is necessary to lock the data structure so that only one thread can modify it during rebalancing
• Without rebalancing, the algorithm can’t do detection• Without synchronization primitives, data plane can
only collect data and detection must be done at the control plane
Schwab_20050928_28
Experiment process using Juniper Experiment process using Juniper routers and CloudShield appliancerouters and CloudShield appliance
1. Allocated via assign into an experiment that put the CloudShield sensor on a GB link between two Juniper routers
2. Router interfaces had to be configured manually3. Routes had to be configured manually4. Configuration scripts were created to help speed
up the experiment setup.5. Routes needed to be verified after setup
completed6. There were issues in determining which physical
CloudShield interfaces were actually being used
Schwab_20050928_29
IntruShield SensorsIntruShield Sensors
• Two McAfee IntruShield Sensors, each with 2 unidirectional Gigabit interfaces and 6 unidirectional 100 MB IPS interfaces have been added to DETER as an experimental device.
• Can be allocated into an experiment by Assign• Usage is complicated by the transparent bridging
function that causes the DETER switches to go into layer 2 loops. (STP issue revisited)
• May require physical attachment to a permanent anchor PC node.
• Requires a dedicated Windows2000 Manager node
Schwab_20050928_30
OverviewOverview
• EMIST DDoS Experimental Methodology
• FloodWatch and CloudShield Experiment Design
• Testbed Methodology Automation
• Process for Incorporating Hardware and Appliances
• Traffic Generation
• Visualization and Metrics
• Future Plans
Schwab_20050928_31
Background Traffic GenerationBackground Traffic Generation
• DDoS Specific Requirements:• Detection and mitigation of attacks need to be
evaluated against a statistically valid mix of background traffic.
• Interactive TCP traffic exhibits very different characteristics due to congestion control and recovery
• Two traffic generation systems under active investigation and development:
– NTCG (UC Davis)– Harpoon (U. Wisconsin)
Schwab_20050928_32
NTCG: NTCG: Network Traffic Compiler Generator Network Traffic Compiler Generator
• Interactive Traffic: Paradigm is to create controlled mix of traffic produced by an array of traffic generators (TG, etc.).
Packet Traceanalyzer
Traffic
Specifications
Topology
Description spec
compiler
TG, etc.
commands
Designed and developed by Allen Ting and colleagues/U.C. Davis
Schwab_20050928_33
Harpoon Traffic GeneratorHarpoon Traffic Generator
• Provides extreme configurability– specify precisely ordered temporal distributions of connection
duration or use Netflow traces– accepts multiple subnets as destination, allowing a single
Harpoon client to generate traffic across all networks– client and server accept either 0.0.0.0 (default routes) or the
IP of a specific interface– Can connect to and listen on all TCP and UDP ports
• Multithreaded implementation– Allows one agent to handle fifty or more connections
(governed by the OS threads per process limit and CPU)• Each thread can connect to any destination, rather than
using a single source / single sink paradigm• Plug-in architecture allows for extensibility• Well documented, clear and precise
Developed by Joel Sommers and Paul Barford, U. Wisconsin
Schwab_20050928_34
IP Address DiversityIP Address Diversity
• Background traffic should involve a wide variety of IP addresses
– There are usually not enough nodes to obtain enough IP diversity
• Solution: Network Address Translation (NAT)– Using Linux IPTables, one machine can serve large blocks of
IP addresses
• Real machine 10.1.1.2 has a single IP address– Use IPTables to add a pre-routing entry to the NAT table– iptables -t nat -A PREROUTING -d 3.0.0.0/8 -j DNAT --to 10.1.1.2
• Packets arriving on any 3.0.0.0/8 are translated into packets destined for 10.1.1.2
• NAT is opaque to 10.1.1.2, allowing full TCP stacks to reply, including proper TCP backoff, etc.
Schwab_20050928_35
Demo Video ClipDemo Video Clip
• An Attack run against the FloodWatch DDoS defense, as performed and recorded by Brett Wilson on DETER.
Schwab_20050928_36
Requirements for NAT (Routed) Requirements for NAT (Routed)
• Traditional network with routes in and out– Start NATs on each of the nodes– iptables -t nat -d 2.0.0.0/8 -j DNAT -A PREROUTING --to 10.1.2.3
• Configure routers so that there are valid routes for the NAT networks.
• At the final hop, include a route to the NAT network (10.1.2.3) through the next hop
– This prevents the router from ARPing for a NAT address, as it believes 10.1.2.3 is another route
– On 10.1.2.3, IPTables catches the packet first (PREROUTING), translates it, and the connection completes
Schwab_20050928_37
Requirements for NAT (LAN) Requirements for NAT (LAN)
• Isolated LAN configuration– More Complex Configuration
• This scenario has no routers; all nodes are on a single broadcast domain
• Configure each node so that the physical interface has a /32 netmask– ifconfig eth0 netmask 255.255.255.255– eth0 will no longer answer ARP requests itself
• Add explicit route for the experiment network via eth0– Use the broadest net possible– route add -net 10.0.0.0 netmask 255.0.0.0 dev eth0
• Add routes for all the NAT subnets on the other machines– route add -net 3.0.0.0 netmask 255.0.0.0 dev eth0
• Proxy ARP for all subnets this node is NATting. ./tarpd eth0 2.0.0.0 255.0.0.0
• Add the NAT via iptables– iptables -t nat -d 2.0.0.0/8 -j DNAT -A PREROUTING --to 10.1.2.3
Schwab_20050928_38
OverviewOverview
• EMIST DDoS Experimental Methodology
• FloodWatch and CloudShield Experiment Design
• Testbed Methodology Automation
• Process for Incorporating Hardware and Appliances
• Traffic Generation
• Visualization and Metrics
• Future Plans
Schwab_20050928_39
Data Capture and AnalysisData Capture and Analysis
• DDoS Specific Requirements:• Automate collection of both packet traces and host-
based statistics– to calculate metrics– to analyze traffic behavior
• Support visualization and interaction between experimenter and potentially large, unwieldy data harvested from the testbed
Schwab_20050928_40
DDoS:DDoS:Instrumentation and VisualizationInstrumentation and Visualization
• Tracefiles produced, collected and extracted from DETER for post-mortem analysis.
• ESVT analysis and experimental data graphical browser:– Navigate within experiment topology.– Drill down on link or node traffic/metrics: visualize and
compare one or more statistics between different links or time intervals.
• Key Milestone: Integrated support for visualization and analysis of DDoS experiments across multiple links, topologies, attack patterns, and defense configurations.
• ESVT integration in collaboration with Jason Hart / Penn State
Schwab_20050928_41
ESVT Experiment TopologyESVT Experiment Topology
Schwab_20050928_42
ESVT VisualizationESVT Visualization
Schwab_20050928_43
ESVT Visualization: ConnectivityESVT Visualization: Connectivity
Schwab_20050928_44
Spectral Analysis RationaleSpectral Analysis Rationale
• Can we create high fidelity testbed experiments? – Gross parameters such as rate, packet type, etc.– Small time scale behavior captured by spectral analysis
• High fidelity experiments help capture accurate interaction with cross traffic.
Schwab_20050928_45
Spectral AnalysisSpectral Analysis
• Periodic behaviors encoded into the attack stream – Host, Network, aggregation effect, ….
• Leverage experience from signal processing– FFTs, wavelets, detection theory, etc.
08:14:33.495046 2005host1.starwoodbroadband.com.domain > 2005host74.starwoodbroadband.com.32894: 40672 2/1/1 CNAME[|domain]08:14:33.495924 2005host74.starwoodbroadband.com.33350 > camembear.fogcreek.net.http: S 1688588990:1688588990(0) win 5840 <mss 1460,sackOK,timestamp 27125396 0,nop,wscale 0> (DF)08:14:33.496502 2005host74.starwoodbroadband.com.32894 > 2005host1.starwoodbroadband.com.domain: 19420+ PTR? 219.120.243.64.in-addr.arpa. (45) (DF)08:14:33.496956 london-bar3.ja.net > 12.160.3.255: icmp: echo request08:14:33.497347 2005host74.starwoodbroadband.com > london-bar3.ja.net: icmp: echo reply
Characterize attack behavior
Schwab_20050928_46
08:14:33.495046 2005host1.starwoodbroadband.com.domain > 2005host74.starwoodbroadband.com.32894: 40672 2/1/1 CNAME[|domain]08:14:33.495924 2005host74.starwoodbroadband.com.33350 > camembear.fogcreek.net.http: S 1688588990:1688588990(0) win 5840 <mss 1460,sackOK,timestamp 27125396 0,nop,wscale 0> (DF)08:14:33.496502 2005host74.starwoodbroadband.com.32894 > 2005host1.starwoodbroadband.com.domain: 19420+ PTR? 219.120.243.64.in-addr.arpa. (45) (DF)08:14:33.496956 london-bar3.ja.net > 12.160.3.255: icmp: echo request08:14:33.497347 2005host74.starwoodbroadband.com > london-bar3.ja.net: icmp: echo reply08:14:33.499372 i.am.an.animal.and.i.irc.from.zoo-gate.fi > 12.160.3.255: icmp: echo request (DF)08:14:33.499441 2005host74.starwoodbroadband.com > i.am.an.animal.and.i.irc.from.zoo-gate.fi: icmp: echo reply…
FFT
packet trace
time-series
frequencydomain
Fourier AnalysisFourier Analysis
• FFT summarizes the frequency content • Used to characterize spatial effects
Schwab_20050928_47
Wavelet AnalysisWavelet Analysis
08:14:33.495046 2005host1.starwoodbroadband.com.domain > 2005host74.starwoodbroadband.com.32894: 40672 2/1/1 CNAME[|domain]08:14:33.495924 2005host74.starwoodbroadband.com.33350 > camembear.fogcreek.net.http: S 1688588990:1688588990(0) win 5840 <mss 1460,sackOK,timestamp 27125396 0,nop,wscale 0> (DF)08:14:33.496502 2005host74.starwoodbroadband.com.32894 > 2005host1.starwoodbroadband.com.domain: 19420+ PTR? 219.120.243.64.in-addr.arpa. (45) (DF)08:14:33.496956 london-bar3.ja.net > 12.160.3.255: icmp: echo request08:14:33.497347 2005host74.starwoodbroadband.com > london-bar3.ja.net: icmp: echo reply08:14:33.499372 i.am.an.animal.and.i.irc.from.zoo-gate.fi > 12.160.3.255: icmp: echo request (DF)08:14:33.499441 2005host74.starwoodbroadband.com > i.am.an.animal.and.i.irc.from.zoo-gate.fi: icmp: echo reply…
wavelet
long packet trace
time-series
frequency and timedomain
• Wavelets summarize both time and frequency information• Used to characterize temporal effects
Schwab_20050928_48
Case #1: Analysis techniqueCase #1: Analysis technique
• Validate fidelity of experiments by comparing spectral behavior of real-world traces to testbed experiments.
Schwab_20050928_49
Experiment 1: Lander Attack
TCP NULL attack from a single source sending 40B packets at 1100 packets/s
Packet rate
Bit rate
Number of Flows
Trace Spectrum
Testbed Spectrum
Schwab_20050928_50
Experiment 1: Lander Attack
TCP NULL attack from a single source sending 40B packets at 1100 packets/s
Testbed Spectrum
Schwab_20050928_51
Packet rate
Bit rate
Number of Flows
Experiment 2: Lander Attack
Invalid IP-proto 255 attack from three sources sending 40B packets at 60 Kpackets/s
Trace Spectrum
Testbed Spectrum
Schwab_20050928_52
Experiment 2: Lander Attack
Invalid IP-proto 255 attack from a three sources sending 40B packets at 60 Kpackets/s
Trace Spectrum
Schwab_20050928_53
Case #2: Background Traffic AnalysisCase #2: Background Traffic Analysis
• CloudShield Entropy detection behavior
• CloudShield requires filter to be installed from control plane with minimum and maximum thresholds
– Packets that push the entropy above [below] a maximum [minimum] threshold are dropped
• What if entropy standard deviation is 23%?– Non-stationary statistic makes filtering impossible
Schwab_20050928_54
Entropy During Attack RunEntropy During Attack Run
0
500000000
1000000000
1500000000
2000000000
2500000000
3000000000
3500000000
4000000000
4500000000
5000000000
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65
Series1
Series2
Series3
Series4
Series5
Schwab_20050928_55
OverviewOverview
• EMIST DDoS Experimental Methodology
• FloodWatch and CloudShield Experiment Design
• Testbed Methodology Automation
• Process for Incorporating Hardware and Appliances
• Traffic Generation
• Visualization and Metrics
• Future Plans
Schwab_20050928_56
Introduction to our DDoS metric frameworkIntroduction to our DDoS metric framework
• Distinguish between extrinsic and intrinsic metrics– Extrinsic: Measures that can be computed and observed by external
parties – Intrinsic: Measures that can only be computed by the object being
measured and by only analyzing the internal algorithms and data structures
• Analyze metrics at different levels of abstraction– packet, flow, aggregate, service/application layers
• Metrics from different vantage points– Client-side, server-side (end point), link-level, end-to-end
• Focus on metrics for – Traffic and service characterization– Measuring DDoS impact– Measuring DDoS effectiveness
Schwab_20050928_57
Examples of Metrics for Characterizing Traffic Examples of Metrics for Characterizing Traffic (base traffic metrics)(base traffic metrics)
Client-observed Intermediary-observed link level
Server-observed
Application-level
- streaming video: mean opinion score (MOS) (E)- VoIP: Round-trip-delay (E)- VoiP: Percentage of Packets discarded by the Jitter Buffer (I)- VoIP: Mean-length-of-bursts (I)
- number-of-flows-per application
- client-request-rate- client-service-rate
Aggregate-level - per-aggregate-packet-rate (E)
- per-aggregate-arrival-rate (E)
Flow-level - server-connection-completion-time (E)- server-connection-completion-rate (E)- rate of failed connections (E)- per-flow-loss rate
- per-flow-packet-rate (E) - per-client-connection- request-rate (E)- per-client-goodput (E)
Packet-level - server-response-rate (E);- server-response-time (E);
- goodput (E);- ratio of attack traffic to goodput
- per-client-packet-rate (E);- packet-drop-rate (I);- per-packet-processing overhead (I)
Schwab_20050928_58
The Server Service and Link Service Indices:The Server Service and Link Service Indices:Higher Level Posture IndicatorsHigher Level Posture Indicators
• Server Service Index (SSI)– a function of common service attributes such as:
» average server-response time, » Rate of failed connections,» max number of concurrent clients serviced,» server throughput
• Link Service Index (LSI)– a function of common link attributes such as:
» goodput,» link utilization,» loss rate
• Index value is a composite weighted measure of deviations from desired thresholds for constituent attributes
• Normalize to lie within [0, 1]– An SSI/LSI value of 0.5 means the server/link is meeting prespecified
objectives for various attributes and deviations from 0.5 indicate a deficiency (if < 0.5) or a surplus (if > 0.5)
• Changes in the SSI/LSI provide an early warning that the server or link is degrading or improving
1
0
Schwab_20050928_59
Formulating the Server/Link IndexFormulating the Server/Link Index
2
),(1 mm
mm txw
I =
),( mm txwhere for each metric m with a weight wm we define a deviation function that measures how the value x of metric m deviates from the threshold tm
),max()(),(
m
mmm tx
txtxsigntx
The deviation function should have the following properties:• should have no units, • should be zero when x = tm
• should be in the interval [-1,0) if the value of the metric has not achieved the threshold and in the interval (0,1] if it has exceeded the threshold
• should reflect the effort needed to take the metric m from the value x to tm
• The SSI or LSI may be defined as:
Sample linear deviation function: where sign(y) is defined as:
Value of y Threshold is Sign(y)
Positive or zero Upper bound -1
Negative Upper bound 1
Positive or zero Lower bound 1
Negative Lower bound -1
Schwab_20050928_60
Metrics for Characterizing Traffic in Our ToplogyMetrics for Characterizing Traffic in Our Toplogy
- Goodput;- Ratio of attack to background traffic- Average link utilization
- Aggregate attack rate
Victim/Server
-Average server response time-Average server-side application throughput
1
0
Link Service Index (LSI)(link posture)
Server Service Index (SSI)(overall service posture)
1
0
-Average connection completion time-Rate of failed connections-Average throughput per flow-Average loss per flow
end
-to-
end
(T
CP
) fl
ow
end-to-end
client to server
Schwab_20050928_61
Metrics for Measuring DDoS ImpactMetrics for Measuring DDoS Impact
-decrease in goodput
- increased aggregate attack rate
Victim/Server
- degraded server response time- decreased server-side application throughput
1
0
-decrease in the Link Service Index (LSI)
- decrease in the Server Service Index (SSI) (overall service posture)
1
0
- increased connection completion time- increased rate of failed connections- increased loss per flow
end
-to-
end
(T
CP
) fl
ow
end-to-end
client to server
Schwab_20050928_62
Metrics for Measuring DDoS Defense Metrics for Measuring DDoS Defense EffectivenessEffectiveness
• % improvement in base and composite metrics
• Time taken to achieve a % improvement• Metrics to characterize the breaking points for
a defense – e.g: a filter can filter bad traffic up to 200 Mbps
• A high level effectiveness score (index) for a defense mechanism
Schwab_20050928_63
Metrics for Measuring Defense EffectivenessMetrics for Measuring Defense Effectiveness
- increase in goodput
- decreased aggregate attack rate
Victim/Server
- improved server response time- increased server-side application throughput
1
0
-increase in the link Service Index (LSI)
- increase in the Server Service Index (SSI) (overall service posture)
1
0
- decreased connection completion time- decreased rate of failed connections- decreased loss per flow
end
-to-
end
(T
CP
) fl
ow
end-to-end
client to server
Schwab_20050928_64
Measuring Defense Effectiveness Measuring Defense Effectiveness Against TCP Flows Under AttackAgainst TCP Flows Under Attack
• Goal:– Measure the effectiveness of FloodWatch defense
mechanisms in isolating the effects of flooding attacks on TCP flows
– Formulate a metric that is tied to a measure of the spread of the throughput of TCP flows when under attack and protected by FloodWatch and in relation to some ideal (theoretic) notion of what the throughput should be if the flows are unaffected by the attack.
» Ideal throughput• f(r,p) = sqrt(1.5) / r (sqrt (p)) ……….eq (1)
where r = round-trip time and p is the loss rate
Schwab_20050928_65
Towards a Defense Effectiveness MeasureTowards a Defense Effectiveness Measure
• Induce loss rates of 1,2,3, and 4% and subject a sample size of n flows to each loss rate (rk), k = 1,2,3,4.
• Calculate the standard deviation (sdk) of the observed throughputs of the flow for each sample with respect to the theoretical throughput fkas given by eq (1)
• Derive an effectiveness score for each loss rate as a function of sdk
ek = function (sdk)
• Derive and effectiveness score for the defense as the mean of all ek for k = 1,2, 3,4.
• The lower the E value, the more effective the defense• Can also normalize E as an index to lie between [0, 1]
k
eE k
k
Schwab_20050928_66
Potential Future Directions in DDoS MetricsPotential Future Directions in DDoS Metrics
• Mapping of attacks to extrinsic metrics– Development of a classification of attacks and analyzing what the
externally perceived effect of an attack category is (in terms of the extrinsic metrics that will be perturbed)
• Mapping of extrinsic to intrinsic metrics– Goal is to understand why a specific type of defense technology is
limited or in some cases completely fails to mitigate certain attacks classes.
• Towards the development of a DDoS Outage Index– Metrics to measure DDoS at Internet scales. – The long-term goal is the development of the equivalent of the
outage index used by telecommunications and cable TV providers to report outages to the FCC.
– Index will be a function of:» Extent of an attack (how global, how many tierX ISPs affected
etc.)» Spread rate» Duration of the attack» Impact: how many critical services were affected?» Recovery time etc.
Schwab_20050928_67
Future Experiments and Future Experiments and Experimental MethodologyExperimental Methodology
• Communicating DDoS defenses– Stress the experimental methodology by requiring network
communication during DDoS attack
• Juniper routers– Support for routing protocols (BGP, OSPF) within experiment
automation framework– Leverage Chris Alfeld’s WAIL router building blocks
• CloudShield Appliance– As programmable instrument– As an emulation of a router line card, e.g. RED queue variants– As development prototyping environment for various defenses