Upload
amie-hunter
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Mitigating the Insider Threat using High-dimensional Search and Modeling
Telcordia Contact:Eric van den Berg(732) 699 [email protected]
An SAIC Company
SRS PI Meeting, Arlington, VA, January 27, 2005
Team:Shambhu Upadhyaya, Hung Ngo (SUNY Buffalo)Muthu Muthukrishnan, Raj Rajagopalan (Rutgers)
SRS PI Meeting – 2
Talk Outline
Project Overview– Goal
– What is done now?
– Technical Approach
– Technical Challenges
– Quality Metrics
– Expected Achievements
– Task schedule and milestones
Project Progress– Design
– Lightweight Demo
SRS PI Meeting – 3
Project overview
Project goal: to build a system that defends critical services and resources against insiders, which– Can detect attacks by correlating large numbers of sensor
measurements– Can synthesize appropriate pro-active responses to protect critical
services while minimizing collateral damage. What is done today?
– Reactive systems: Detect attacks late in cycle– Anomaly detection systems: Few streams for correlation, suffer from
curse of dimensionality– Human-in-the-loop systems: Response not scalable, prior attacks
pulled from administrator experience– Consequences of response v.s. impact attack: collateral damage
may be large
SRS PI Meeting – 4
Project overview (continued)
Technical Approach– Large network of sensors, to let insider trigger alerts– High dimensional network state description in terms of sensor alerts– Search engine finds top-K historical states similar to sensor snapshot– Insider modeler and analyzer tool used to identify attack points, train search
engine, guide sensor placement– Response engine to analyze impact of potential attack on critical services
and synthesize reconfiguration response
Technical Challenges– Extensive experience with SVD based searches in text-based information
retrieval. Here we are testing search technology in a new domain– New ‘Insider analyzer’ key-challenge graph problem is hard– Training search engine, labeling and annotating states
SRS PI Meeting – 5
Project overview (continued)
Quantitative Metrics to measure success and overheads– Keep track of detection performance: detection/false alarm rate
– Test detection for novel attacks which are variations of known attacks
Expected Major Achievements– New high-dimensional anomaly / intrusion detection system, which
can scale to large, internet-size networks
Task schedule and milestones
Task(milestone) Jan-Jun 05
Jul-Dec 04
Design (document) Prototyping (software) Testing (report)
Jul-Dec 05
SRS PI Meeting – 6
Proposed architecture
Sensor
Sensor
Sensor
Sensor
Normalizer
Filter
Aggregator
SearchEngine
ResponseEngine
Network StateRepository
HostScans
AuditScans
StateData
TrafficMeasure-
ments
Reconfiguration
Top KList
RefinedQueriesN
ETWORK
High-DimensionalSearch
Insider Modeler
andAnalyzer
Organizationaldata
Labels andfilters for states
Post-processing
SRS PI Meeting – 7
Sensor network design
Monitor critical services and applications, hosts, devices on which these depend:– Network sensors: aggregate traffic, network flows, histogram-
based, ‘sketch’-based.– Host sensors: applications, cpu-load, audit logs, web logs, user
challenges, profile anomalies, file-integrity checkers Design goals: scalability and extensibility
– Incorporate available sensors as well as newly developed sensors easily
Use data-aggregation and filtering to reduce data volume– Only store sensor alerts
Store sensor alerts in unified format– IDMEF-like database schema to store sensor data
SRS PI Meeting – 8
Network state description
Network state is constructed from sensor alerts:– Accommodate heterogeneous sensor types
– Account for different sensitivity of sensor types
– Tolerate possibly delayed or missing, ‘out of order’ alerts
Alerts are mapped to a high-dimensional vector for search– Coordinates correspond to different sensor-alert types
– Some possibilities for mapping values: Total number of sensor alerts of given type in (sliding) time window Indicator: sensor alert occurred in (sliding) time window
Network state is labeled:– With Classification e.g. ‘Normal’, ‘DoS’, ‘Insider’
– With Response for Response Engine
SRS PI Meeting – 9
Search engine design
Goal: Find historical documented network states most similar to the current network state snapshot
Output: Top-K list of ranked/prioritized similar states Ranking can be based on similarity metric and/or potential
impact, e.g. attack ‘risk’. Impact of historical network states is documented, impact of
current state can be analyzed/verified with the Response engine
Search engine reduces dimensionality of search space– Using Singular Value Decomposition, or random projection
Similar states found by nearest neighbor search using distance metric (e.g. cosine similarity, Euclidean distance)
SRS PI Meeting – 10
Normal
InsiderDoS
Worms
High dimensional Search on Labeled States
1.0
S81.0
S13
S14
S4
S10
S15
S9
query
S1S2
S6
S16
S5S11
S12
S7S17
S3
SRS PI Meeting – 11
Precursors: early detection of insider attacks
All attacks, including Insider attacks, need to be detected as early in the cycle as possible to minimize damage
Nearest neighbor search and/or cluster analysis may help label and diagnose vector-based network states
– but how to represent time evolution? Like learning attacks from documented historical network states, we can
also document attack precursors or attack stages – Full attack now represented as a sequence of network state vectors– Initially, we can obtain attack ‘cases’ from forensic analysis– Robust against slow attacks: no explicit dependence on time– Would like to make ‘precursor’ annotation (semi-) automatic
Two possible approaches to automatic precursor annotation– Temporal precursors: use (e.g. exponential) temporal decay functions leading
up to attacks to indicate confidence in network state as precursor– Spatial precursors: consider all state vectors occurring within a time window of
length T of an attack vector to be ‘correlated’, I.e., precursors.
SRS PI Meeting – 13
Impact Analysis using Response Engine
Building upon Smart Firewalls technology from Dynamic Coalitions program
– Response Engine has overview of current network configuration– Response Engine logically validates Policies, expressed in terms of end-to-
end service availability– Response Engine generates candidate reconfigurations to comply with
Policies as much as possible In this project
– Detected attack type and location is translated into its effect on the stated policies and current network configuration
– E.g. Server failure due to a Denial of Service attack Response Engine can analyze the impact of both the attack and its
candidate responses on the availability of critical resources– E.g. Analyze impact of vulnerability exploit: how widespread is the
vulnerability? Administrator can push response into the network
SRS PI Meeting – 15
Insider analyzer and modeler
Insider threat manifests in two forms:
– Insider abuse while staying within legitimate privileges– Insider abuse while exceeding assigned privileges
Focus on an insider's view of an organization such as hosts, reachability and access control
A new threat model called a “key challenge graph”
– Similar to attack graphs, but far less emphasis on details– Allows static analysis of insider threat on problem instances of manageable
size
Threat analysis metricCost of AttackActual targetTarget Vertex
Location of insiderStarting VertexAccess ControlKey Challenge
Information, CapabilityKey
Connectivity, ReachabilityEdgeHosts, PeopleVertex
AbstractionModel Component
SRS PI Meeting – 16
Features of key challenge graph
Ease of representation– Unlike attack graphs, KCG mirrors actual network topology– Abstract info. in the form of hosts, people, channels, access control etc.– (May lose some accuracy, but it is a time-tradeoff)
Proactive scheme – Threat assessment– Works as a precursor to our online detection subsystem– Can also be used independently
Answers important questions such as:– Is the current security set-up sufficient?– If not, what are likely attack paths? – Additional security recommendations? What parts of organization need
additional monitoring? Which security policies need to be revised? Role and impact:
– Guide the placement of sensors for data collection– Cluster weighting and size reduction
SRS PI Meeting – 19
Detection of an insider attack via honeytokens
Insider Fin-data
CFOAdmin
Login: cfo @ fin-data
Password: ”makemoney”
Login: cfo @ fin-data
Password: ”makemoney”
Login: cfo @ fin-data
Password: ”makemoney”
ALARM!
Sensor 1 (NIDS)
Sensor 2 (HIDS)
SRS PI Meeting – 20
Alert-id 1 Sensor-id 1 “Honeytoken access”
NORMAL
Alert-id 3 Sensor-id 2 “Honeytoken used”
Alert-id 2 Sensor-id 1 “Honeytoken access”
Alert-id 1 Sensor-id 1 “Honeytoken access”
Search engine detection
Network State Repository:
prior network statesand
responsesALARM!
Alert-id 2 Sensor-id 2 “Honeytoken used”
Alert-id 1 Sensor-id 1 “Honeytoken access”
ATTACK
Search Engine