Upload
gabriel-coyle
View
215
Download
3
Tags:
Embed Size (px)
Citation preview
A Technical Evaluation and Critique of:
“Techniques and Tools for Analyzing Intrusion Alerts”
by Peng Ning, Yun Cui, Douglas S. Reeves, and
Dingbang Xu
A Technical Evaluation and Critique of:
“Techniques and Tools for Analyzing Intrusion Alerts”
by Peng Ning, Yun Cui, Douglas S. Reeves, and
Dingbang Xu
Angela OrebaughIT862
4/28/05
Angela OrebaughIT862
4/28/05
Formal Framework
Framework Highlights
• Correlates alerts on the basis of prerequisites and consequences of attacks
• Matches the consequences of some prior alerts with the prerequisites of some later ones
• Constructs attack scenarios
• Represented in Hyperalert Correlation Graph
• Uses nodes to represent alerts and edges to represent the relationships between the alerts.
Framework Notation• Prerequisite Predicate
• UDPVulnerableToBOF(VictimIP, VictimPort)
• Consequence Predicate
• {GainRootAccess (VictimIP), rhostsModified (VictimIP)}
• Logical combination of predicates for complex attacks
• UDPVulnerableToBOF(VictimIP, VictimPort) ^ (UDPAccessibleViaFirewall (VictimeIP, VictimPort)
Framework Notation (2)• HyperAlert Type T (fact, prerequisite,
consequence)
• SadmindBufferOverflow = ({VictimIP, VictimPort}, ExistHost (VictimIP) ^ VulnerableSadmind (VictimIP), {GainRootAccess (VictimIP)})
• HyperAlert Instance h
• hSadmindBOF = {(VictimIP = 152.1.19.5, VictimPort = 1235), (VictimIP = 152.1.19.7, VictimPort = 1235)}
• ExistHost (152.1.19.5) ^ VulnerableSadmind (152.1.19.5), ExistHost (152.1.19.7) ^ VulnerableSadmind (152.1.19.7)
• GainRootAccess (152.1.19.5), GainRootAccess (152.1.19.7)
Hyperalert Correlation
• In a sequence S of hyperalerts, a hyperalert h is...
• Correlated hyperalert
• if there exists another hyperalert h’ in S such that either h prepares for h’ or h’ prepares for h.
• Isolated hyperalert
• if no such h’ exists
Hyperalert Correlation Graph
Additional Utilities• Aggregation/Disaggregation
• All hyperalerts of type FTP-BOF combined• All hyperalerts that are DoS are combined
• Focused Analysis• SrcIP = 192.168.1.1 V DestIP = 192.168.20.1
• Clustering Analysis• (A1.SrcIP = A2.SrcIP) ^ (A1.DestIP = A2.DestIP)
• Frequency Analysis• Counting the number of raw alerts that share the same
destination IP address to fnd the most frequently hit target
• Link Analysis• How two IP addresses are realted to each other in a
collection of alerts
• Association Analysis• Many attacks are from source IP 152.14.51.14 to destination
IP 129.14.1.31 at destination port 80
TIAA(Toolkit for Intrusion Alert
Analysis)
TIAA Architecture
Most Recent TIAA Software
• http://discovery.csc.ncsu.edu/~pning/software/correlator
• Current version 0.4
• Tested on Windows 2000 and XP with MS SQL Server
• Newly added features
• Association Analysis (Extracting frequent coourrences of attribute values from a set of alerts)
• Attack Strategy Extraction (Extracting attack strategies from a correlation graph)
• Missed Attack Hypotheses (Hypothesizing possibly missed attacks)
TIAA Knowledge Base<hyper-alertType Name="SadmindOverflow">
<Fact FactName="DestIPAddress" FactType="varchar(15)"></Fact> <Fact FactName="DestPort" FactType="int"></Fact><Protocol ProtocolName="RPC"></Protocol> <Protocol ProtocolName="SADMIND"></Protocol> <Prerequisite>
<Predicate Name="ExistHost"> <Arg id="3" ArgName="DestIPAddress"></Arg>
</Predicate> <Predicate Name="VulnerableSadmind">
<Arg id="22" ArgName="DestIPAddress"></Arg> </Predicate>
</Prerequisite> <Consequence>
<Predicate Name="GainRootAccess"> <Arg id="18" ArgName="DestIPAddress"></Arg>
</Predicate> </Consequence>
</hyper-alertType>
Experiments
Experiments
• 2000 DARPA intrusion detection dataset
• Aimed at evaluating the effectiveness of the proposed alert correlation method in constructing attack scenarios and its ability to differentiate true and false alerts.
• DEFCON 8 CTF
• Intended to evaluate the usefulness of the analysis utilities in dealing with large collections of intrusion alerts.
DARPA Dataset• LLDOS 1.0 - Series of attacks in which an attacker
probes, breaks in, installs the components necessary to launch a DDoS attack, and launches a DDoS attack against an off-site server.
• LLDOS 2.0.2 - Similar sequence of attacks by a more sophisticated attacker.
• Each dataset contains network traffic collected from both the DMZ and the internal network.
• Testing used 4 sets of experiments, each with either the DMZ of the inside network traffic of one dataset.
DEFCON 8 CTF Dataset
• Capture the flag contest
• Attacks range from script kiddie to sophisticated attacker
• Largest graph had 2,940 nodes and 25,321 edges
• On average each graph had 21.75 nodes and 310.56 edges
DARPA Experiment Results
• TIAA revealed the structure and high-level strategy of the sequence of attacks
• Real Secure generated duplicate alerts for several attacks
• Correlated a few false alerts• ISS >93% false alert rate• TIAA reduced to 5% for LLDOS 1.0 and 23%-40% for LLDOS
2.0.2
• Correlated normal alerts that were not attacks
• Missed the Telnet portion of the attack
• LLDOS 2.0.2 results were unsatisfactory
DEFCON Experiment Results
• Probably some missed alerts - So many attacks occurring at once
• Alert aggregation reduced the largest graph to 77 nodes and 347 edges
• 7 clear stages of attacks
• Utilities helped discover several attack strategies
• Scanning attacks followed by attacks that may lead to execution of arbitrary code
• Not good for forensics
Related Work
First Class of Approaches• Staniford 2002
• Probability distribution for normal traffic to detect portscan attacks
• SPICE/SPADE
• Valdes and Skinner 2001• Mathematical framework for correlating alerts that match
closely but not perfectly• EMERALD
• Cuppens 2001• Alert clustering and merging via expert system approach• Also uses pre and post attack conditions based on LAMDA• MIRADOR project
• Julisch 2001• Alarm clustering to determine root causes• Alarm clustering and summarizing
Second Class of Approaches
• Eckmann 2002• State transition based attack scenarios• State Transition Analysis Technique Language (STATL)
• Cuppens and Ortalo 2000• Attack scenarios with pre and post conditions using
the LAMBDA attack description language
• Debar and Wespi 2001• Detects duplicates and consequences according to
explicit rules• Built on top of Tivoli Enterprise Console
Third Class of Approaches
• Templeton and Levitt 2000• Capabilities/concepts attack model describes unknown attacks
and predict attacker actions• JIGSAW attack specification language
• Cuppens and Miege 2002• CRIM module based on LAMBDA to cluster, merge, and
correlate alerts• MIRADOR project
• Morin 2002• M2D2 data model for correlation• Uses correlation function to detect false positives
• Porras 2002• Mission-impact-based approach• MCorrelator uses an internal topology map for correlation
• Peng Ning, et. al. 2003
Vulnerability Analysis Approach
• Ritchey and Ammann 2000• Modeling based approach based on host vulnerability, host
connectivity, current point of view of attacker, exploits that can change the state of the model
• Uses a state machine to encode the vulnerabilities
• Sheyner 2002• Automated technique for generating and analyzing attack
graphs• Based on intruder preconditions, network preconditions,
intruder effects, and network effects
• Jha 2002• Expands on Sheyner paper• Presents a formal and detailed explanation of the model• Presents an algorithm to compute the reliability for a
network
Summary
Major Contributions
• Hyperalert correlation graphs
• Partial satisfaction of attack prerequisites
• Uses possible consequences instead of actual consequences
• Analysis Utilities
• TIAA
Framework Critique
• Successfully revealed relationships between alerts and strategies behind the attacks
• Effectively reduces the number of alerts via aggregation
• Needs to address partial satisfaction of prerequisites more thoroughly
• It does not address security architecture• It doesn’t address network issues• IDS evasion• May not discover stealthy and intelligent attacks• Can still be evaded
TIAA Critique• Needs to provide more information on the
Knowledge base• Does the accuracy of the system all come down to the
robustness of the knowledge base?
• “the results produced by our correlation techniques are only as good as the hyperalert information provide by the user”
• Only supports IDMEF and ISS Real Secure alerts• http://www.sans.org/rr/whitepapers/detection/
1080.php
• Only works with a commercial database - MS SQL Server
• Not meant for the inexperienced use
Testing Critique
• Uses ISS Real Secure
• Are the DARPA and DEFCON CTF datasets the best methods of testing?• DARPA dataset has received lots of criticism
• DEFCON CTF is all attack traffic and not much else
• What is the best testing approach anyway?• Test network, live network, replay data taken from
other networks, DARPA or other pre-generated datasets
• Tested DARPA datasets separately
Architecture• Does not address a recommended security architecture for
optimization
• Sensor placement is critical to correlation
Firewall w/logs
Desktopw/HIDS and AV
Web
IDS
Management Station
DNS
Database
Router w/logs
IDS
IDS
Internet
Additional Research Opportunities
• Expanded use of hyperalert correlation graphs• Attacker profiling• Predictive analysis• IDS tuning• Input to incident response procedures
• Incorporate forensic analysis
• Integrate framework with complementary correlation methods for better performance
• Use TIAA as part of penetration testing team
• Automatic generation of knowledge base by learning algorithm
Discussion...
• Critiques of the framework?
• Critiques of the TIAA toolkit?
• Critiques of the testing?
• Additional research/expansion opportunities?