CSC 382: Computer SecuritySlide #1 CSC 382: Computer Security Network Monitoring

CSC 382: Computer Security Slide #1

CSC 382: Computer Security

Network Monitoring


Topics

1. Principles

2. Models of Intrusion Detection

3. False Positives

4. Architecture of an IDS

5. IDS Deployment

6. Active Response (IPS)

7. Host-based IDS and IPS


Principles of Intrusion Detection

Characteristics of systems not under attack1. User, process actions conform to statistically

predictable pattern.

2. User, process actions do not include sequences of actions that subvert the security policy.

3. Process actions correspond to a set of specifications describing what the processes are allowed to do.

Systems under attack do not meet at least one.


Example

Goal: insert a back door into a system– Intruder will modify system configuration file

or program.– Requires privilege; attacker enters system as an

unprivileged user and must acquire privilege.• Nonprivileged user may not normally acquire

privilege (violates #1).• Attacker may break in using sequence of commands

that violate security policy (violates #2).• Attacker may cause program to act in ways that

violate program’s specification (violates #3).


Goals of IDS

1. Detect wide variety of intrusions– Previously known and unknown attacks.

– Need to adapt to new attacks or changes in behavior.

2. Detect intrusions in timely fashion– May need to be be real-time, especially when system

responds to intrusion.

• Problem: analyzing commands may impact response time of system.

– May suffice to report intrusion occurred a few minutes or hours ago.


Goals of IDS

3. Present analysis in easy-to-understand format.– Ideally a binary indicator.– Usually more complex, allowing analyst to examine

suspected attack.– User interface critical, especially when monitoring

many systems .

4. Be accurate– Minimize false positives, false negatives.– Minimize time spent verifying attacks, looking for

them.


Deep Packet Inspection

• IDS requires, some firewalls do too.

• DPI = Analysis of Application Layer data

• Protocol Standard Compliance– Is port 53 traffic DNS or a covert shell session?– Is port 80 traffic HTTP or tunneled IM or P2P?

• Protocol Anomaly Detection– Traffic is valid HTTP.– But suspicious URL contains directory traversal.


Models of Intrusion Detection

1. Anomaly detection– What is usual, is known.– What is unusual, is bad.

2. Misuse detection– What is bad is known.– Look for what is bad, hope it doesn’t change.


Anomaly Detection

Analyzes a set of characteristics of system, and compares their values with expected values; report when computed statistics do not match expected statistics.– Threshold metrics– Statistical moments– Markov model


Threshold Metrics

• Counts number of events that occur– Between m and n events (inclusive) expected– If number falls outside this range, anomalous.

• Example– Windows: lock user out after k failed sequential

login attempts. Range is (0, k–1).• k or more failed logins deemed anomalous

• Threshold depends on typing skill.


Sequences of System Calls

• Define normal behavior in terms of sequences of system calls.

• Example normal trace:open read write open write close

• Doesn’t normally run other programs.

• Attack trace:open read write open exec write close


Finding Features

• Which features best show anomalies?– CPU use may not, but I/O use may.

• Use training data– Anomalous data marked.– Feature selection program picks features,

clusters that best reflects anomalous data.

• Use compiler techniques to build program model.


Misuse Detection• Determines whether a sequence of instructions

being executed is known to violate the site security policy.– Descriptions of known or potential exploits grouped

into rule sets.– IDS matches data against rule sets; on match, potential

attack found.

• Cannot detect new attacks:– No rules to cover them.


Example: snort

Network Intrusion Detection System– Sniffs packets off wire.– Checks packets for matches against rule sets.– Logs detected signs of misuse.– Alerts adminstrator when misuse detected.


Snort Rules• Rule Header

– Action: pass, log, alert– Network Protocol– Source Address (Host or Network) + Port– Destination Address (Host or Network) + Port

• Rule Body– Content: packet ASCII or binary content– TCP/IP flags and options to match– Message to log, indicating nature of misuse detected


Snort Rule Example

Example: rule for ssh shell code exploit

alert tcp $EXTERNAL_NET any -> $HOME_NET 22 (msg:"EXPLOIT ssh CRC32 overflow NOOP"; flow:to_server,established; content:"|90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90|"; reference:bugtraq,2347; reference:cve,CVE-2001-0144; classtype:shellcode-detect; sid:1326; rev:3;)


Comparison and Contrast• Misuse detection: if all policy rules known, easy

to construct rulesets to detect violations.– Usual case is that much of policy is unspecified, so

rulesets describe attacks, and are not complete.

• Anomaly detection: detects unusual events, but these are not necessarily security problems.


False Positives

• A new test for a disease that is 95% accurate

• Assume 1 in 1000 people have disease.

• Should everyone get the test?– Sample size: 1000– Expect 0.95 + (999 * 0.05) positives– Ergo, 50 people will be told they have disease – If you test positive, only 2% chance you have it.


IDS Architecture

An IDS is essentially a sophisticated audit system– Agent gathers data for analysis.

– Director analyzes data obtained from the agents according to its internal rules.

– Notifier acts on director results.

• May simply notify security officer.

• May reconfigure agents, director to alter collection, analysis methods.

• May activate response mechanism.


Agents

Obtain information and sends to director.

Preprocessing– Simplifying and reformatting of data.

Push vs Pull– Agents may push data to Director, or– Director may pull data from Agents.


Host-Based Agents

1. Obtain information from logs– May use many logs as sources.

– May be security-related or not.

– May use virtual logs if agent is part of the kernel.

2. Agent generates its information– Analyzes state of system.

– Treats results of analysis as log data.


Network-Based Agents• Sniff traffic from network.

– Use hubs, SPAN ports, or taps to see traffic.– Need agents on all switches to see entire network.

• Agent needs same view of traffic as destination– TTL tricks, fragmentation may obscure this.

• End-to-end encryption defeats content monitoring– Not traffic analysis, though.


Aggregation of Information

Agents produce information at multiple layers of abstraction.– Application-monitoring agents provide one

view of an event.– System-monitoring agents provide a different

view of an event.– Network-monitoring agents provide yet another

view (involving many packets) of an event.


Director

• Reduces information from agents– Eliminates unnecessary, redundent records.

• Analyzes information to detect attacks– Analysis engine can use any of the modelling

techniques.

• Usually run on separate system– Does not impact performance of monitored systems.

– Rules, profiles not available to ordinary users.


Example

• Jane logs in to perform system maintenance during the day.

• She logs in at night to write reports.• One night she begins recompiling the

kernel.• Agent #1 reports logins and logouts.• Agent #2 reports commands executed.

– Neither agent spots discrepancy.– Director correlates log, spots it at once.


Adaptive Directors

• Modify profiles, rulesets to adapt their analysis to changes in system– Usually use machine learning or planning to

determine how to do this.

• Example: use neural nets to analyze logs– Network adapted to users’ behavior over time.– Used learning techniques to improve

classification of events as anomalous.• Reduced number of false alarms.


Notifier

• Accepts information from director

• Takes appropriate action– Notify system security officer– Respond to attack

• Often GUIs– Use visualization to convey information.


Example Architecture: snort


IDS Deployment

IDS deployment should reflect your threat model.

Major classes of attackers:1. External attackers intruding from Internet.2. Internal attackers intruding from your LANs.

Where should you place IDS systems?1. Perimeter (outside firewall)2. DMZ3. Intranet4. Wireless


IDS Deployment


Sguil NSM Console


Intrusion Prevention Systems

• What else can you do with IDS alerts?– Identify attack before it completes.– Prevent it from completing.

• How to prevent attacks?– Directly: IPS drops attack packets.– Indirectly: IPS modifies firewall rules.

• Is IPS a good idea?– How do you deal with false positives?


IPS Deployment Types

Inline

IPSIntranet

Non-Inline

Intranet

IPS


Active Responses by Network Layer

• Data Link: Shut down a switch port. Only useful for local intrusions. Rate limit switch ports.

• Network: Block a particular IP address.– Inline: can perform blocking itself.

– Non-inline: send request to firewall.

• Transport: Send TCP RST or ICMP messages to sender and target to tear down TCP sessions.

• Application: Inline IPS can modify application data to be harmless: /bin -> /ben


Host IDS and IPS

• Anti-virus and anti-spyware– AVG anti-virus, SpyBot S&D

• Log monitors– swatch, logwatch

• Integrity checkers– tripwire, osiris, samhain

• Application shims– mod_security


Key Points

• Intrusion detection is a form of auditing.• Models of IDS:

– Anomaly detection: unexpected events.

– Misuse detection: violations of policy.

• The problem of false positives.• IDS Architecture:

– Agents.

– Director.

– Notifiers.

• Active Response: Intrusion Prevention Systems


References1. Richard Bejtlich, The Tao of Network Security Monitoring, Addison-Wesley, 2004.2. Matt Bishop, Computer Security: Art and Science, Addison-Wesley, 2003.3. Brian Caswell, et. al., Snort 2.0 Intrusion Detection, Snygress, 2003.4. William Cheswick, Steven Bellovin, and Avriel Rubin, Firewalls and Internet

Security, 2nd edition, 2003.5. The Honeynet Project, Know Your Enemy, 2nd edition, Addison-Wesley, 2004.6. Richard A. Kemmerer and Giovanni Vigna, “Intrusion Detection: A Brief History and

Overview,” IEEE Security & Privacy, v1 n1, Apr 2002, pp 27-30.7. Steven Northcutt and Julie Novak, Network Intrusion Detection, 3rd edition, New

Riders, 2002.8. Michael Rash et. al., Intrusion Prevention and Active Response, Syngress, 2005.9. Rafiq Rehman, Intrusion Detection Systems with Snort: Advanced IDS Techniques

Using Snort, Apache, MySQL, PHP, and ACID, Prentice Hall, 2003.10. Ed Skoudis, Counter Hack Reloaded 2/e, Prentice Hall, 2006.11. Ed Skoudis and Lenny Zeltser, Malware: Fighting Malicious Code, Prentice Hall,

2003.

Documents

CSC 382: Computer SecuritySlide #1 CSC 382: Computer Security Network Monitoring