View
221
Download
0
Tags:
Embed Size (px)
Citation preview
Real-Time Intrusion Detection with Emphasis on Insider Attacks
Shambhu Upadhyaya Computer Science and Engineering
University at Buffalo
Polytechnic University
October 3, 2003
CEISARE @2
Some Facts & Figures (CSI/FBI 03)
CEISARE @3
Source of Attack
CEISARE @4
Attack Types
CEISARE @5
Actions Taken
CEISARE @6
What Could be Learned From It?
Good prevention techniques must be in place
Good policies must be set up
Need to know what is important
Need to know the application environment
IDS is a must
But there is no IDS that is applicable to all environments
CEISARE @7
Outline of the Talk
General introduction
Evolution of IDS
Major players
Insider threats and how to mitigate?
Conclusion
CEISARE @8
Outline of the Talk
General introduction
Evolution of IDS
Major players
Insider Threats and how to mitigate?
Conclusion
CEISARE @9
What is an IDS?
In its general sense –
Acquires information about its environment to analyze
system behavior
Aims to discover security breaches, attempted breaches,
open vulnerabilities that could lead to potential breaches
Types of information –
Long term info. – a knowledge base of attacks (static)
Configuration info. – a model of the current state (static)
Audit info. – describing the events happening (dynamic)
CEISARE @10
IDS Architecture(Macroscopic View)
System Model ofSystem
Analyzer Visual Presentation
Database,Storage
DATA
Slide adopted from UCDavis, Jeff Rowe
CEISARE @11
IDS Side-effects
False negatives (failed detection)
poor coverage
False positives (wrong indictment)
poor QOS
Degrade normal operation
poor performance
CEISARE @12
Outline of the Talk
General introduction
Evolution of IDS
Major players
Insider Threats and how to mitigate?
Conclusion
CEISARE @13
Evolution of IDS
Paul Innella’s timeline:
CEISARE @14
Current State-of-the-art
1st generation tools are largely signature based
Security is by penetrate and patch
Today’s focus is on detecting novel intrusions
New techniques must consider insider attacks, social
engineering based break-ins etc.,
Need for new paradigms – Design for Security?
New ideas –
Combining IDS with vulnerability analysis
Detection is not fool-proof; must be merged with recovery
CEISARE @15
Outline of the Talk
General introduction
Evolution of IDS
Major players
Insider Threats and how to mitigate?
Conclusion
CEISARE @16
Major Players – Academia Purdue –
CERIAS
UC Davis –
Developed GrIDS (Graph based IDS)
CMU – Home of CERT/CC
Cornell
Language-based security
Columbia
IDS and Data mining
Above list is incomplete
CEISARE @17
Major Players – Industries
IBM Watson
Global Security Analysis Laboratory
Microsoft
Started the Trustworthy Computing initiative in 2002
Cisco
Does research and development
Builds intrusion detection appliances – sensors and software
MAFTIA
European Union of academia and industries
CEISARE @18
Major Players – Labs/Government SRI International –
Developer of EMERALD through funds from ITO, DARPA
Air Force Research Lab –
Defensive Information Warfare Branch
Naval Research Lab –
Center for High Assurance Computer Systems
Multi-level security
National Institute of Standards and Technology –
Computer Security Resource Center
National Security Agency –
Research and education
CEISARE @19
Popular Websites
SANS (System Administration, Networking and Security) Institute
http://www.sans.org/aboutsans.php
CERT/CC
http://www.cert.org/
CERIAS (Center for Education and Research in Information
Assurance and Security)
http://www.cerias.purdue.edu/
NIST (National Institute of Standards and Tech.)
http://csrc.nist.gov/index.html
CEISARE @20
IDS Tools List Mike Sobirey (copyright: Dr. Michael Sobirey)
List of ID Tools from 1995-2000
92 host- and network based Intrusion Detection (&
Response) Systems
Additions are appreciated
NIST Intrusion Detection Tools
Coverage is only up to 1996 (not up-to-date)
About 20+ tools listed
The above two lists have little overlap (cover >110)
CEISARE @21
Recent Releases Responsible for real-time packet capture and analysis (http://www.prelude-ids.org/) on Linux/Unix
Prelude platforms
Portsentry – An IDS that detects and responds to port scans against a target host in real-
time (http://www.psionic.com/products/)
SPADE – Statistical Packet Anomaly Detection Engine (http://www.silicondefense.com/)
inspects recorded data for anomalous behavior based on a computer score
Stealthwatch (Lancope), Stormwatch (Okena)
Stackguard – Protects from stack smashing attacks (
http://www.cse.ogi.edu/DISC/projects/immunix/StackGuard/)
Netscreen -- http://www.netscreen.com/products/
There is no tool that is universally applicable
CEISARE @22
Outline of the Talk
General introduction
Evolution of IDS
Major players
Insider Threats and how to mitigate?
Conclusion
CEISARE @23
Who is an Insider?
CEISARE @24
How to Deal with the Problem?
We focus on the detection only
Model the Insider
Prevention of Insider Misuse
Detection, Analysis and Identification of Misuse
CEISARE @25
IDS with Emphasis on Insider
Current systems are signature-based and they
use audit-trail or rule-based protection
Not effective for insider attack detection
Anomaly detection is applicable, but not very
effective
New theory needed, proactive mechanisms
needed
CEISARE @26
Guidelines for Effective Anomaly Detection
Use the principle of least privilege to achieve better
security
Use mandatory access control wherever appropriate
Data used for intrusion detection should be kept
simple and small
Intrusion detection capabilities are enhanced if
environment specific factors are taken into account
CEISARE @27
Our Approach An out-of-the-box Reasoning Framework for intrusion
detection
Technique used:
Control flow checking from FT (basis for encapsulation of
owner’s intent)
Reasoning based on Theory of Risk Analysis from
Economics
Problem is similar to Pricing Under Uncertainty S. Upadhyaya, R. Chinchani, K. Kwiat, “An Analytical Framework for Reasoning about Intrusions”,
IEEE SRDS 2001
CEISARE @28
User Intent Encapsulation
Obtain the intent of the
user either by inference
or query
Session scope serves as
a certificate
Reduces the search
space during monitoring
CEISARE @29
Illustration of Search Space Reduction
Kernel
Resources
Commands and System
calls
Audit data
reduces as we go
higher up Typical audit data
User
CEISARE @30
InterfaceUser
Sequence of Operations
Resource
Disk Network
CPU Memory
System
Overall Layout of System Operation
CEISARE @31
Expected Sequences
Certain “normal” ways of doing a job
Also, certain “less normal” ways of doing them
A job is completed by performing a sequence of
operations
May not be possible to enumerate all the
sequences
CEISARE @32
Cost Analysis
Cost of Operation = Co
Proportional to the amount of resources used
Cost of Sequence = Cd
Proportional to the difference between current
chosen operation and past history
Cost of Job = *Co + *Cd
CEISARE @33
Job Activity Stochastic
At any stage, a user “chooses an operation”
with a probability
“Choice of an operation” is a random variable
Sequences of operations construct a discrete
stochastic process
CEISARE @34
User Activity as a Martingale
Theorem:
Let the lateral sequence of random
variables for any state i of a sequence of
operations be denoted as:
X1(ti, ), X2(ti, ), … Xn(ti, )
Such a sequence of user activity is a
Martingale
CEISARE @
An Example
nfrm pine exit nop nop
pine ls exit nop nop
mail finger nfrm pine exit
nfrm pine finger exit nop
(nfrm, pine, ls, mail, exit, finger, nop)
CEISARE @36
A Note on Martingale
Martingale uses concepts of conditional probability and
has applications in economics
Model is used to predict market parameters like a share of
a stock
Future price of a commodity depends only on the last
known distribution and not on the entire history of the
prices
There is a parallel between uncertainties in intrusion
detection and the concept of pricing under uncertainty
CEISARE @37
Reasoning
Non-intrusive
Non-deterministic
Intrusive
Monotonically increasing costs
ThTl
CEISARE @38
Cost Scenarios
Low Co + Low Cd
Non-intrusive
Maps into the non-intrusive region
High Co + Low Cd
Intrusive and tending toward a DoS attack on
resources
Maps into the non-deterministic region
CEISARE @39
…contd.
Low Co + High Cd
The intruder??
Maps into the non-deterministic region
High Co + High Cd
The clumsy attack
Maps into the intrusive region
CEISARE @40
Quantification of Thresholds
Threshold Tl
Minimum cost over longest sequence
Threshold Th
Maximum cost over shortest sequence
CEISARE @41
Algorithm: INIT_DISTR(Generates the initial distribution)
Enumerate all possible
sequences
Find the longest sequence
Create a discrete stochastic
process
Generate probabilities at
each stage and shape the
distribution
CEISARE @42
Algorithm: MODIFY_DISTR(Modifies the existing distribution)
Check to see at each stage of the
sequence if the user is conforming to the
profile
At the job termination, if the sequence is
not intrusive, update the frequency
distributions and probabilities
CEISARE @43
Algorithm: DECIDE(Makes a decision in the non-deterministic region)
Calculate the longest sequence from
current stage to complete the job. Move Tl
to that position
The window (Th – Tl) depends on the
gradient of the cost accumulated since
DECIDE was last invoked
CEISARE @44
Sketch of the Overall Algorithm User logs into the system
Chooses the job s/he wishes to performCheck the size of the session scope
If too large,warn userUser wants to change it
Launch inter work-space level monitor
Create workspaces for the jobs
Launch workspace level monitor thread per workspaceLaunch command level monitor thread per command
Authenticate command
Monitor Command
YES
LoopReport command type
Report object accessed
CEISARE @45
Preliminary Implementation
Developed in Java on Solaris 2.8
A university environment was simulated
Monitoring at basic command level
Limited sequence monitoring
Not many scenarios
Perhaps, not realistic for actual deployment
CEISARE @46
Test Cases
User activity collected over two months
Test cases grouped into four categories
1-user, 1-user with multiple logins, multiple users, multiple users
with multiple logins
Two sets of experiments – worst case and average case
Legitimate and intrusive operations
32 attacks
Obvious ones such as transferring /etc/passwd files, exploiting
vulnerabilities such as rdist, perl 5.0.1
Subtle attacks similar to mimicry attacks
CEISARE @47
Summary of Results
Summary 1 User, No Multiple Logins 1 User, With Multiple Logins 2 Users, No Multiple Logins 2 Users, With Multiple LoginsUser Detection 87.50% 78.60% 74.90% 91.90%and Latency 33.4 35 36.1 29User False Positives 12.50% 21.40% 25.10% 8.10%
False Negatives 0% 0% 0% 0%User Detection 98% 89% 100% 94.70%and Latency 0 11 0 9.6
Intruder False Positives 0% 0% 0% 0%False Negatives 2% 11% 0% 5.30%
Intruder Detection 99% 100% 98.20% 100%and Latency 0.4 0.7 0.6 0.5User False Positives 0% 0% 0% 0%
False Negatives 1.40% 0% 1.80% 0%Intruder Detection 56% 81.30% 77.40% 91.50%
and Latency 15.9 14.8 17 27Intruder False Positives 0% 0% 0% 0%
False Negatives 44% 18.70% 22.60% 8.50%
CEISARE @48
Types of Detected Intrusions
It can detect internal attacks -
A cracker logs in and executes commands
Inadvertent operator faults
Internal abuse
External attacks -
Masquerading
Subversion attacks by presenting overly
permissive session-scope (penalty in terms of
reduced QoS)
CEISARE @49
Undetected Intrusive Activity
It cannot contain or detect -
External Denial of Service attacks
Extremely low-level network based attacks
CEISARE @50
Ongoing Research
Addressing outstanding issues like
State explosions due to partial orderings
Scalability
Values of α, β , ??
A more realistic prototype implementation and
testing
Project is currently funded by DARPA
CEISARE @51
Concluding Remarks – Vision Insider threat is very much real
Penetrate and Patch method is not adequate
CMU and other studies show current IDS are not effective
Anomaly detection schemes, that are environment-independent
may be in the focus
Monitoring at user command level has distinct advantages
Conceptually independent of systems and applications
As the no. of threats grows, IDS will become a required element
of system security
CEISARE @52
Concluding Remarks – Research IDS and vulnerability analysis
Effective means of system evaluation
Good metrics for performance, coverage etc.
Return on investment studies
Merge IDS with firewalls
Merge IDS with recovery
It is not possible to detect all intrusions
Protection against unknown threats – Proactive mechanisms
Rapid Incident Response