View
221
Download
2
Embed Size (px)
Citation preview
1
Intrusion Detection Intrusion Detection Alert CorrelationAlert Correlation
Mark Shaneck
2/11/2005
2
Outline
Problem Statement Different Correlation Approaches A Comprehensive Approach Good News and Bad News A Better Approach?
3
What’s The Problem?
Large organizations get tons of alerts Possibly up to 20,000 per day! Many false alarms
4
Also…
Alerts can come from many different sources– Signature based IDS (Snort)– File System Integrity Checkers– System Call Traces
Alerts may represent multiple stages in one attack Hard to make sense out of a large pile of alerts!
5
So What Is Alert Correlation?
3 general categories– Alert Clustering– Matching Predefined Attack Scenarios– Prerequisites/Consequences
6
Alert Clustering
Main Sources:– A. Valdes, K. Skinner, “Probabilistic Alert
Correlation”, RAID 2001– O. Dain, R. Cunningham, “Building Scenarios
from a Heterogeneous Alert Stream”, IEEE Workshop on Information Assurance and Security, 2001
7
General Idea
Join alerts together in some meaningful groups
Group alerts into attack threads - one thread contains all alerts related to one attack
For a new alert, compare to all alert threads– Join to the closest match – Or start new thread if none match
8
Similarity Measure
Feature Overlap - only consider features present in both (source, target, ports, attack class, timestamps, etc.)
Each feature has a similarity measure– How much do port lists overlap?– Is one port contained within another’s list? (target port
was previously scanned)– Are the IPs from the same subnet?– Attack classes have a similarity matrix
9
Similarity Expectation
Different levels of similarity are expected for different features in different situations– SYN FLOOD with source spoofed
• Expectation of similarity for source IP is 0
– Scanning port(s)• Expectation of target IP is low (but not 0 - since it
usually scans the subnet)
10
Minimum Similarity
Threshold for similarity measure Similarity is 0 if not above the minimum
Adjusting thresholds– Synthetic Threads
• high for sensor id, IPs
– Security Incidents • low for sensor id, high for attack class• fuse alerts from multiple sources
– Multistep attack detection • low for attack class
11
So What Is Alert Correlation?
3 general categories– Alert Clustering– Matching Predefined Attack Scenarios– Prerequisites/Consequences
12
Matching Predefined Attack Scenarios
Main sources– H. Debar, A. Wespi, “Aggregation and
Correlation of Intrusion-Detection Alerts”, RAID 2001
– B. Morin, H. Debar, “Correlation of Intrusion Symptoms : an Application of Chronicles”, RAID 2003
13
Aggregation and Correlation
Correlation– Group alerts that are part of the same attack trend– Duplicates– Consequences (chain of related alerts)
Aggregation– Group alerts based on certain criteria to aggregate
severity level, reveal trends, clarify attacker’s intentions
– Situations
14
Duplicates
Duplicates Definition– Initial Alert Class– Duplicate Alert Class– List of Attributes (that must be equal)– Severity Level (new severity level for new
merged alert)
Specified by analyst
15
Consequences
Consequences Definition– Initial Alert Class– Initial Probe Token– Consequence Alert Class– Consequence Probe Token– Severity Level– Wait Period
Links together alerts that are sequential in nature
16
Aggregation
Aggregate based on three axes– Alert Class– Source– Target
Putting wildcards for different cases gives different views
Aggregate into scenarios
17
Scenarios
Same source/target/attack class– A single attacker launching attacks against a single
victim Same source/destination
– Single attacker running many attacks on a single victim Same target/attack class
– Distributed attack against a single victim Same source/attack class
– A single attacker running the same attack against multiple victims
18
Chronicles
“Set of events, linked together by time constraints, whose occurrence may depend on the context”
Similar to plan recognition Used to model known attack “chunks”
– Long attack scenarios may have many paths – Certain small sequences of events almost
certainly occur together
19
So What Is Alert Correlation?
3 general categories– Alert Clustering– Matching Predefined Attack Scenarios– Prerequisites/Consequences
20
Prerequisites/Consequences
F. Cuppens, A. Miège, “Alert Correlation in a Cooperative Intrusion Detection Framework”, In IEEE Symposium on Security and Privacy, 2002
P. Ning, D. Reeves, et al. (many papers)– Check my website for the list– Or the very last slide…..
21
Prerequisites/Consequences
Prerequisite: the necessary condition for the attack to be successful
Consequence: the possible outcome of the attack
Represented as a logical formula– Using only AND and OR connectives
22
Hyper Alert Type
(fact, prerequisite, consequence) SadmindBufferOverflow =
({VictimIP, VictimPort},
ExistHost(VictimIP) AND VulnerableSadmind(VictimIP)
{GainRootAccess(VictimIP)})
23
Prepare-For Relationships
An alert “prepares for” another alert if it contributes to the second alert’s prerequisite set
Also must occur earlier in time
24
Correlation Graph
Directed acyclic graph, with the nodes being alerts and the edges being the prepares-for relations
Could be huge!
25
Adjustable Reduction
Aggregation of alerts of the same type Can result in overly simple graphs Adjustable
– Analyst can specify a time interval– Only alerts with time gap less than the interval
are merged
26
Adjustable Reduction
27
Focused Analysis
Logical combination of comparisons between attribute names and constants
SrcIP = 129.174.142.2 OR DestIP = 129.174.142.2
Useful for focusing on a critical server
28
Graph Decomposition
Cluster alerts based on “common” features Use clusters to separate large graph into
smaller ones (A1.SrcIP = A2.SrcIP) AND (A1.DestIP = A2.DestIP)
Clustering constraints are specified by the analyst
29
Reduced and DecomposedGraph Example
30
Matching Attack Strategies
Attack Strategy Graph– Set of events linked together by certain
constraints• Time Order
• IP Addresses
Events can be generalized to deal with variations
SadmindBufferOverflow
TooltalkBufferOverflowRPCBufferOverflow
31
Measuring Similarity Between Attack Strategies
Error Tolerant Graph Isomorphism Use edit distance to derive a similarity
measure Can be used to find similar attacks or to
match against predefined strategies
32
Hypothesizing About Missed Attacks
Missed attacks can break up the graphs– One attack graph becomes two disconnected,
seemingly unrelated, attack graphs Indirect Prepares-for Similarity based merging of attack graphs Prune hypotheses with network traffic
– E.g. one hypothesized attack is ICMP ping, but no ICMP traffic occurred during that time
33
Outline
Problem Statement Different Correlation Approaches A Comprehensive Approach Good News and Bad News
34
A Comprehensive Approach
F. Valeur, G. Vigna, C. Kruegel, R. Kemmerer, "A Comprehensive Approach to Intrusion Detection Alert Correlation", In IEEE Transactions on Dependable and Secure Computing, 2004
35
Alert Fusion
Combine alerts that are independent detection of the same attack instance– Must be temporally close– From different sensors– Identical overlapping attributes
36
Alert Verification
Idea: False positives can negatively impact alert correlation
Filter out false positives and irrelevant positives (alerts that correspond to failed attacks)
37
Alert Verification
Passive: use network knowledge to see if attack could succeed (low overhead, low confidence)– Listing of existence of/services running on IPs– Firewall configurations
Active: check for evidence (high overhead, high confidence)– See if service is still running and available– See if extra ports are open– Use vulnerability scanner to test target machine– Remote login and run scripts
38
Thread Reconstruction
Group alerts that refer to attacks launched by one attacker against a single target
Merge alerts with same source and destination and within a time interval
39
Attack Session Reconstruction
Link network based alerts to host based alerts
Manually specify links between network events and process events– Alert on web server process (or one of its
children) can be correlated to a (temporally) nearby network alert targeted to that machine on port 80
40
Focus Recognition
Identify hosts that are the source or target of lots of attacks
Merge these alerts together into one Source: Scanning Target: DDoS
41
Multi-Step Correlation
Identify attack patterns that are made up of multiple individual attacks
Create attack patterns by means of expert knowledge
Simply match the merged alerts to the attack strategies
42
Experimental Results
Defcon9– Input: 6,378,096 alerts– Output: 203,303 alerts– Reduction: 96.81%
TreasureHunt– Input: 2,811,169 alerts– Output: 1,080 alerts– Reduction: 99.96%
MIT/LL 2000– Input: 36,635 alerts– Output: 17,220– Reduction: 53.00%
43
Benefits of Alert Correlation
Higher level representation of alerts reduces clutter and can show attack structure
Reduce false positives– False positives are unlikely to correlate with
other alerts May find many attacks and respective
scenarios
44
Limitations of Correlation
Relies on IDS to alarm each step of the attack– Exploit mutations– Novel attacks– Bad sensor placement– Sensor overload - packet loss– Restricted ruleset for better performance
Relies heavily on a priori expert knowledge
45
Limitations of Correlation (cont)
Cannot provide a comprehensive view on network attacks
46
MINDS Level 2
Level 1 IDS alerts Anchor Point Identification Context Extraction Attack Characterization Behavior/Host Profiling
47
Questions?
Paper links located at:http://www.cs.umn.edu/~shaneck/wormlist.html– At the bottom of the page
Slides available:http://www.cs.umn.edu/~shaneck/Correlation.ppt
48
A Budding Hacker
49
Peng Ning Reference List
1. P. Ning, D. Reeves, Y. Cui, "Correlating Alerts Using Prerequisites of Intrusions", Technical Report, TR-2001-13, North Carolina State University, Department of Computer Science, December 2001
2. P. Ning, Y. Cui, D. Reeves, "Analyzing Intensive Intrusion Alerts via Correlation", In Recent Advances in Intrusion Detection, 2002
3. P. Ning, Y. Cui, D. Reeves, "Constructing Attack Scenarios through Correlation of Intrusion Alerts", In CCS 2002
4. P. Ning, D. Xu, "Learning Attack Strategies from Intrusion Alerts", In CCS 20035. P. Ning, D. Xu, C. Healey, R. St. Amant, "
Building Attack Scenarios through Integration of Complementary Alert Correlation Methods", NDSS, February 2004
6. Y. Zhai, P. Ning, P. Iyer, D. Reeves, "Reasoning about Complementary Intrusion Evidence", 20th Annual Computer Security Applications Conference, December 2004
7. D. Xu, P. Ning, "Alert Correlation Through Triggering Events and Common Resources", 20th Annual Computer Security Applications Conference, December 2004
8. P. Ning, D. Xu, "Hypothesizing and Reasoning about Attacks Missed by Intrusion Detection Systems", ACM Transactions on Information and System Security, 2004