Cortex Program Status Brief Christopher Geib December 14th 2005

Cortex Program Status Brief

Christopher GeibDecember 14th 2005

2 PIMeetingStatusBrief.ppt

Talk Outline

• Quick Program Overview.• Technical approach.• Accomplishments since last PI Meeting.• Red Team Results.• Metrics.• Next Steps/Follow-on directions.


Self Regeneration that is Mission Sensitive

• Systems must be designed that are capable of response and regeneration after deliberate attack and catastrophic failures.

• However, response and regeneration must be sensitive to…- the mission that is being executed,

What tools and services are critical for the current mission goals? What tools and services are not critical for the current mission goals?

- and the lessons learned from the previous failure. What features of the protocol were exploited in the attack? Have features of the domain changed?

• Are there new kinds of connections that should be blocked?• Are some kinds of attacks more common than we thought?


Program Research Objectives

• Prove the viability of automatically synthesizing a meta-level controller for model-based, system response and regeneration.

• Such regeneration control systems must have two critical capabilities:- Response planning that is sensitive to the system mission,

How much of the systems resources should be committed or held in reserve?

Planning conditional responses to known threats. Trading off commitments for mission critical objectives.

- and learning to prevent similar failures in the future. Patch the services that were exploited Modify the mission model to capture changes to the world.

Architecture and Technical Approach


Program Level Demonstration Objectives

• Control self-regeneration of mission critical service.• Scenario of use Air Tasking Order generation.• Protect a MySQL DB, guaranteeing availability.

TastersTastersTaster DBReplicatorRTS.

PlannerOnce per phase

Proxy (Dexter)

Current LeadTaster DB

Input Query Stream

Learner

Domain Model


Planning Approach: Generalized Semi-Markov Decision Processes

• Most existing decision-theoretic planning systems are based on the Markov Decision Processes and have difficulty handling multiple, asynchronous events.

• GSMDP provides the most natural framework for this purpose.

• CIRCA (Cooperative Intelligent Real-Time Control Architecture) is the first GSMDP planner.

Uses the decision-theoretic principle of maximizing expected utility (e.g. to trade off performance against safety).

Uses rich stochastic models of world, transitions, actions, and time to construct best defense plans.

Does not hand-build, but automatically synthesizes plans and can thus adapt to defend against combinations/mutations of existing exploits based on the mission model.


Learning Approach

• Active experimentation based on identified axis of vulnerability.- Query content, Query length (single/multiple terms)- Resource consumption patterns- Probing (e.g. password guessing)- Session-wide (multiple queries)

• Active experimentation should avoid false positives.

Build Model of Normal Traffic

Use model to identify suspicious elements

ExperimentHistorical(Normal)Queries

Attack Query

LearnerBlocking

rules and

attack models

Model of

Normal


Risks and Mitigations

• Search space for the controller is large.- Mitigation: Our work is focused on heuristics that are

solving these problems without covering the whole space.- Mitigation: Plans are built offline before system commission.- Mitigation: Possible to build plans that provide safe reduced

functionality states to move to while regenerating. • Identification of a covering set of axes of

vulnerability and experimentation methods.- Mitigation: Fixed protocols have limited degrees of freedom

making it easier to enumerate.- Mitigation: Axis only has to be identified once .

• “Binary poisons”- Mitigation: Don’t eat the second half of the poison. - Mitigation: Use program diversity tools to push some code

level violations that would be binary poisons into our space.

Recent Progress


Infrastructure Progress

• New two stage mission model defined.- Removed Apache and rebuilt to define 2 phase mission.- Building demonstration in progress.

• Extended set of attacks.- 2 Old attacks.

Lion – password buffer overflow. My_Dumper – casting vulnerability for signed integers.

- 2 New attacks identified and implemented. Hoagie – MySQL_shutdown() doesn’t check authority. Eleven – 11 simultaneous connections to MySQL causes crash.

• Independence of MySQL version verified by using earlier version.- Our development done with MySQL 3.23.49.- Tested with earlier version MySQL 3.23.8.


• Classes of exploits:

• Generalization across axis of vulnerability:- Anecdotal evidence for this.- System learned My_Dumper when exposed to Hoagie.

• Addition of learned exploits to attack ontology, and domain threat model.

• No identified false positives.

• Red Team Goals.

Learning Progress

Can’t sense or block (eleven) Can sense, will block (Hoagie) Can sense, block, and generalize

(Lion, MyDumper)


Red Team Results

• Sandia Red Team verified learning capabilities.- Effectively blocked Red Team’s versions of our attacks,

even when presented in close proximity.- Generalized attacks as well.

• Identified (and patched) some weaknesses in RTS and Proxy.- RTS Designed for highly engineered real time systems that

are not subject to variability from human inputs.

• System suffered significant false positive problem.- In some cases blocking all significant access beyond

connection to the MySQL server. - Why? Murphy’s Law: we provided the wrong code

development branch to the red team for testing.


Retesting on Our Own.

• Using the Red Teams versions of the exploits we reran their tests. Got no false positives.

(show movie)

• Testing multiple identical instances of attacks- Inter-attack timing < 1 second results in no connection.- Inter-attack timing > 1 second attack is blocked.

• Haven’t had time to look at some possibilities- “varying the attack ahead of Cortex.”- “bias the model of normal traffic.”

• Very interested in having the Red Team back out.


Load Testing with Background Traffic

• Working on testing against SQLBench, a standard benchmark data set.- Command set: Covers space of SQL commands.- Would be nice to conclusively show no false positives.- Current problem is stock proxy (DEXTER) doesn’t handle it.

• Testing against our own traffic generators- Command set: Inserts, Selects.- Does learn, and will block later instances.- We do loose a few queries. Proxy doesn’t buffer.- Learning happening on the same machine (not required)- Working on looking at the learning load


Effect on System Runtime

Total Runtime for 500 Queries

0100020003000400050006000

Test Condition

Mill

isec

onds

NavopsCommanderAegis1Aegis2


Program Level Metrics

• Correctly diagnosing root cause 10% of attacks.- System correctly diagnoses 50% of collected attacks.- System will diagnose 100% of the attacks covered by an

identified axis of vulnerability.- Question becomes one of coverage of the axes of

vulnerability relative to the set of attacks.

• Correctly responding to 5% of diagnosed attacks- CORTEX plans controllers that respond to all (100%) of the

possible attacks enumerated within the mission model.- Learning blocking rules for 75% of the observed attacks.- Learning generalized blocking rules for 50% of the observed

attacks (those covered by identified axis).


Project Metrics

• Time to synthesize optimal controller (measured)- Experiments have cut the time to find the optimal plan for

this problem by >90%.

• Accuracy of the rules learned (measured)- Limited scope testing against our traffic generators 0% false

positives.- Working on standard SQLBench set for confirmation.

• Overhead added to normal processing (measured)- Tests against our traffic generators

3 x with no hostile traffic 5 x with hostile traffic and limited learning resources.

- Working on standard SQLBench set for confirmation


Expected Major Achievements

• Prove the viability of automatically synthesizing a meta-level controller for model-based system response and regeneration.- Limited impact on users- Over all improved system throughput from increased

system mission sensitive availability.- Demonstrate system scalability in two mission phases.

• Prove the viability of automatically learning previously unseen exploits based on identified axis of vulnerability.

• Demonstrate system planning for at least two different mission phases with significantly different requirements and responses.

Future Directions


Measuring Capacity for Self-Regeneration

• Compare the protected system to unprotected (or differently protected) systems- For the same set of realistic tasks or relevant application

benchmark application.

• Quantify trade-off between system throughput and increased regenerative capacity - for a given hardware platform.

• Baseline performance before attacks- Employ as many types of attacks as available...

• Compare to performance during and after attacks.


Metrics for Phase II

Pre-Attack Attack Post-Attack

CORTEX

Unprotected

Overhead

Maximum degradation

Recovery latency

Tran

sact

ions

Per

Min

ute

Baseline

Illustrative curves—not real data


Metrics for Phase II

• Baseline scale is application specific.- In database application: transactions/unit time.

• Overhead.- Ratio of protected to unprotected performance.

• Recovery Latency.- Time required to restore the system to a threshold

fraction of baseline performance after an active attack has ceased (per attack tested).

• Maximum Degradation.- The ratio of the worst performance to nominal

performance (per attack tested).


Possible Dramatic Performance Increases

• Recoding of Proxy and possibly RTS.- Proxy is currently single threaded Perl.- Retool/Rewrite while retaining extensibility for detection and

control.- RTS not designed for vagaries of human input.

• Correct distribution of system components.

• Improved Learning.

• Improved Planning.


Future Work: Learning

• Research Issues:- More Comprehensive

Sensing. Must be able to detect subtle

forms of attack. Attacks that are not immediately

(or ever?) fatal.

- Automated learning of Axis of Vulnerability. Rely on having a set of sensors

that covers the space. Identify the sensor(s) that are

critical. Determining the

experimentation methodology.- Automated restructuring of

the domain model.

• Benefits:- Defense against wider suite

of more subtle attacks.- Enable faster system

deployment.- Lower cost system

maintenance.


Future Work: Planning

• Research Issues:- Automated Abstraction and

Problem Space Decomposition Finding fully connected (or maybe

weakly connected) subspaces that can allow the system to solve subproblems.

• Some existing work in the DTP. Replanning at runtime.

- Automated use of specialized planning search heuristics.

• Utility based• Resource utilization.• “Sampling” search

- “Counter Planning” for intelligent adversaries Conditionalizing probability of

opposition actions based on our actions.

• Benefits:- Automated response

planning for larger domains.

- Speed for replanning for RTSy.

- More accurate response to intelligent adversaries


SCHEDULE

CORTEX – Mission-Aware Closed-Loop Cyber Assessment and Response

• System Reference Model drives intrusion assessment, diagnosis, and response.• Automatically search for response policies that optimize tradeoff of security against mission ops.• “Taste-tester” server redundancy supports robustness and learning from new attacks.

• High confidence intrusion assessment and diagnosis.• Pre-planned automatic responses to contain and recover from faults and attacks.• Automatic tradeoffs of security vs. service level & accessibility.• Learns to recognize and defeat novel attacks.

Computing services

Active Security ControllerExecutive

Controller Synthesis ModuleNetworks, Computers

Attacks, intrusions

IMPACT

NEW IDEASSecurity Tradeoff Planner

Scyllarus Intrusion

Assessment

Demos:Thin slice

demo

Jan 05 Jul 05

Planning Demo

Dec 05

Mission Aware Learning and

Response Demo

May 05

Learning Demo

End

Documents

Cortex Program Status Brief Christopher Geib December 14th 2005