1

Network-based Intrusion Detection, Mitigation and

Forensics SystemYan Chen

Department of Electrical Engineering and Computer Science

Northwestern University

Lab for Internet & Security Technology (LIST)

http://list.cs.northwestern.edu

2

The Spread of Sapphire/Slammer Worms

3

Current Intrusion Detection Systems (IDS)

• Mostly host-based and not scalable to high-speed networks– Slammer worm infected 75,000 machines in <10

mins– Host-based schemes inefficient and user dependent

• Have to install IDS on all user machines !• Mostly simple signature-based

– Cannot recognize unknown anomalies/intrusions– New viruses/worms, polymorphism

4

Current Intrusion Detection Systems (II)

• Statistical detection – Unscalable for flow-level detection

• IDS vulnerable to DoS attacks

– Overall traffic based: inaccurate, high false positives

• Cannot differentiate malicious events with unintentional anomalies– Anomalies can be caused by network

element faults– E.g., router misconfiguration, link failures, etc.

5

Network-based Intrusion Detection, Mitigation, and Forensics System

• Online traffic recording [SIGCOMM IMC 2004, INFOCOM 2006, ToN to appear]– Reversible sketch for data streaming computation– Record millions of flows (GB traffic) in a few hundred KB– Small # of memory access per packet– Scalable to large key space size (232 or 264)

• Online sketch-based flow-level anomaly detection[IEEE ICDCS 2006] [IEEE CG&A, Security Visualization 06]– Adaptively learn the traffic pattern changes – As a first step, detect TCP SYN flooding, horizontal and

vertical scans even when mixed• Online stealthy spreader (botnet scan) detection

[IWQoS 2007]

6

Network-based Intrusion Detection, Mitigation, and Forensics System (II)

Integrated approach for false positive reduction• Polymorphic worm signature generation & detection

[IEEE Symposium on Security and Privacy 2006]

[IEEE ICNP 2007 to appear]

• Accurate network diagnostics [ACM SIGCOMM 2006] [IEEE INFOCOM 2007]

• Scalable distributed intrusion alert fusion w/ DHT[SIGCOMM Workshop on Large Scale Attack Defense 2006]

• Large-scale botnet event forensics using honeynet[work in progress]

7

System ArchitectureRemote aggregatedsketchrecords

Streaming packet data

Part IIPer-flowmonitoring & detection

Reversiblesketch monitoring

Filtering

Sketch based statistical anomaly detection (SSAD)

Local sketch records

Sent out for aggregation

Per-flow monitoring

Normal flows

Suspicious flows

Intrusion or anomaly alarms

Keys of suspicious flows

Keys of normal flows

Data path Control pathModules on the critical path

Signature-based detection

Polymorphic worm detection

Part ISketch-basedmonitoring & detection

Modules on the non-critical path

Network fault diagnosis

8

System Deployment• Attached to a router/switch as a black box• Edge network detection particularly powerful

Original configuration Monitor each port

separately

Monitor aggregated

traffic from all ports

Router

LAN

Internet

Switch

LAN

(a)

Router

LAN

Internet

LAN

(b)

HPNAIDMsystem

scan

po

rtsc

an

port

Splitter

Router

LAN

Internet

LAN

(c)

Splitter

HR

AID

syst

em

Switch

Switch

Switch

Switch

Switch

HPNAIDMsystem

HPNAIDMsystem

Detecting Stealthy Spreaders Using Online Outdegree Histograms

Yan Gao1, Yao zhao1, Robert Schweller1, Shobha Venkataraman2, Yan Chen1,

Dawn Song2 and Ming-Yang Kao1

1. Northwestern University

2. Carnegie Mellon University

10

Outline

• Motivation• Problem definition• System design• Evaluation• Conclusion

11

Motivation• High-speed network monitoring

– Small amount of memory usage– Small number of memory accesses per packet

• Superspreaders vs. Stealthy spreaders– Superspreaders: sources that connect a large

number of distinct destinations• e.g. a compromised host doing fast scanning for worm

propagation

– Stealthy spreaders: a number of sources that send more than a certain number of connections (unsuccessful) to distinct destinations

• e.g. botnet scans or moderate worm propagation

12

Existing Data Streaming Algorithms

• Online entropy estimation approachesChakrabarti et al. [STACS 06] and Guha et al. [ACM SODA 06]– Pros: detect unexpected changes in the network traffic– Cons: lose some concrete distribution information

• Online histogram estimation algorithms Gibbons et al. [VLDB 97] and Gilbert et al. [STOC 02]

– Pros: provide more information on the features of network traffic– Cons: cannot record the number of unique items

• Superspreader detection schemesVenkataraman et al. [NDSS 05] and Zhao et al. [IMC 05]– Pros: detect sources with an very large outdegree– Cons: memory usage unscalable to small/medium outdegrees

such as bot scansSuperspreader detection is a special case of spreader detection

n

ii

m

m

m

mH

1

)log(

13

Outline

• Motivation• Problem definition• System design• Evaluation• Conclusion

14

Problem Definitions

Two high-level problems

• Construct an approximation of the outdegree histogram online

• Directly detect the presence of stealthy spreaders without constructing the complete outdegree histogram

15

Problem Definition

• Input: stream of (Src, Dst) pairs S• Output

z --- of which powers define the buckets of the histogram (z=2)

…20 21 22 23 24 25 26 27 …

Histogram

Nu

mb

er

of s

ou

rce

s

Number of unique destinations

16

Problem Definition

• Input: stream of (SIP, DIP) pairs S• Output

Wi --- the set of sources

A source s is in Wi if and only if the number of unique destinations that s connects to is in the range of [zi, zi+1)

…20 21 22 23 24 25 26 27 …

Histogram

Nu

mb

er

of s

ou

rce

s

Number of unique destinations

17

Problem Definition

• Input: stream of (SIP, DIP) pairs S• Output

…20 21 22 23 24 25 26 27 …

Histogram

Nu

mb

er

of s

ou

rce

s

Number of unique destinations

mi = |Wi|

Creating an approximate histogram is to estimate mi for each bucket

18

Contribution• Study the problem of detecting stealthy

spreaders online– With constant small memory – With small memory accesses per packet

• Design the algorithm to detect stealthy spreaders online by approximating the outdegree histogram– Data recording phase

• Sampling and coupon collection-based algorithms

– Spreader detection phase• Linear regression to find bins where attacks happen

• Show that the change of approximated histogram reveals the presence of anomalies

19

Outline

• Motivation• Problem definition• System design• Evaluation• Conclusion

20

Recording Phase:

Sampling AlgorithmFast: update a smaller number of counters

per packet

S0 S1 S2 S3 Sd

(src, dst)

Packet

2-3≤ h(src) ≤ 2-2

src

src

src

Sampling algorithm

21

Recording Phase:

Coupon Collecting AlgorithmAccurate: create a better approximation

interim structure

S0 S1 S2 S3 Sd

(src, dst)

Packet

2-3≤ h(src) ≤ 2-2

(src

,g0(

dst)

)

(src

,g1(

dst)

)

(src

,g2(

dst)

)

(src

,g3(

dst)

)

(src

,gd(

dst)

)

Coupon collecting algorithm

(dst)gi : uniform random hash function for hashing dst to an integer in [1, 2i]

22

0m0,l ii

(2) di0for,)ε)(1ε'(1

ll

)ε)(1ε'(1

l

.z)z

1(1z)

z

1-(1][bB

,z)z

1(1z)

z

1-(1][aA

(1) where M,ALMB

i

ii

i

i

j

i

j-1

iij

1j

i

j

iij

• Outdegree histogram constructionInterim data structure -> final outdegree histogramUsing linear programming method

• Build a convex hull

Other constraints: • Find the lower and upper bounds for mi

• Solution– Directly use the interim data structure

Pros: Obtain a reasonably accurate histogram for normal network traffic

Cons: Fail to accurately estimate the outdegree histogram for anomalous traffic

Spreader Detection Phase

23

System Design• Change detection

– The change of the interim data structure of two time intervals

• Stealthy spreader detectionki

’ > ch (threshold)

• System architecture

i2,i1,'i kkk

Current CDCReal

traffic stream Current

SDC

Current

UDCAttack

detectionOnline

histogramcomputing

Recording phase Detection phase

DetectionResults

24

Spreader Detection Phase• The real scan event

Number of distinct destination

Num

ber

of s

cann

ers

One Peak

Close to 0

25

Spreader Detection Phase• Linear regression for coupon collecting

algorithm– Mean squared error as the fitting metric

BucketExample of linear regression

Val

ue o

f cou

ntin

g

26

Outline

• Motivation• Problem definition• System design• Evaluation• Conclusion

27

Evaluation Methodology• Traffic traces

– OC-48 CAIDA data on Aug. 14th, 2002– The average packet rate: 191K/s– The average flow rate: 3.75K/s

• A real scanning event collected from one class B honeynet on Jan 7th, 2007– Port 23– 2.5 hours– 1,607 unique sources– 1,700,236 scan sessions

• Synthetic scanning traces

28

Simulation Results

• Synthetic stealthy scan

Estimate ratio

The estimate ratio of scan outdegree

Per

cent

age

of d

etec

tion

resu

lts

False negative: 17.8%The estimation error within 20%: 33.9%

False negative: 0The estimation error within 20%: 76.1%

Estimate ratio = outdegreespreaderReal

outdegreeEstimated

Attack intensity = trafficnormalofflowsTotal

flowsscanTotal

29

• Synthetic stealthy scan

Simulation Results

Estimate ratio

CDF of estimate ratio for spreader intensity estimation

Cum

ulat

ive

perc

enta

ge (

%)

35%

80%

30

Simulation Results

• Real stealthy scan

Number of distinct destinationThe histogram of outdegree of scanners collected in the honeynet

Num

ber

of s

cann

ers

Estimation: 90

Ground truth: 87

31

Simulation Results

• Real stealthy scan

Estimate ratio

CDF of estimate ratios of scan outdegree estimation

Cum

ulat

ive

perc

enta

ge (

%)

80%

Mix the 5-min data of a real scanning event with 5-min normal traffic of CAIDA data (distribution over 30 such intervals)

32

Online Performance• Memory consumption

– Our method: O(c log(m)) • Constant memory: 24×1KB ＝ 24KB

– Superspreader: • When k is small, the memory usage is closer to the size of

the entire data stream N.

• Memory access per packet– Single memory access per packet for each distinct

counting structure– Speed up: processing in parallel or in pipeline

• Speed– 3.2GHz Pentium 4 computer– Recording: 200 seconds for each 5-min CAIDA data

interval– Detection: less than 0.1 second

)δ1

lnkN

O(

33

Conclusion

• Propose the stealthy spreader detection problem

• Design an online outdegree histogram based stealthy spreader detection algorithm– Propose two randomized algorithms for

recording phase– Propose the linear regression based

approach for stealthy spreader detection