Learning, Monitoring, and Repair in Application Communities

Martin Rinard

Computer Science and Artificial Intelligence Laboratory

Massachusetts Institute of Technology

Goal

Structure of implemented systemHow it works

Planned developments for future

Basic Idea

• Community learns invariants that are always true in successful executions

• Community is attacked• Find a set of invariants that are

violated when attack happens• Deploy several alternative repairs that

enforce violated invariants• Community tries the different repairs,

recognizes which ones work• Successful repairs distributed across

community

System Operational Modes

• Invariant Learning Mode• Monitoring Mode (detecting attacks)• Invariant Localization Mode

(detecting which invariants are violated)

• Protection Mode (deploying and evaluating repairs)

• Modes can be temporally and spatially overlapped

Invariant Learning Mode Architecture

Tracing

Client Library

Determina MPEE

Application

Local Daikon

NodeManager

Central Daikon

ManagementConsole

InvariantDatabase

Trace Data

InvariantsInvariants

Invariants

Invariant Updates

(https/ssl)

Community Machine

Server Machine

What Is Trace Data?

• Sequence of observations<basic block, binary variable, value>

• Binary variables• Variable at binary (not source) level• Type determined by use

• Example1: mov edx, [eax]2: cmp edx, [ecx+4]

• Five binary variables – • 1:eax (ptr) 1:[eax] (int) • 2:edx (int) 2:ecx (ptr) 2:[ecx+4] (int)

Determina MPEE andClient Library

Application (binary)

Basic Block CheckingAnd Transformation

Basic Block

Checked, Transforme

d Basic Block

Code Cache

PC

• In learning mode• Basic blocks are

transformed to print out trace data

Invariant Learning Mode Architecture

Tracing

Client Library

Determina MPEE

Application

Local Daikon

NodeManager

Central Daikon

ManagementConsole

InvariantDatabase

Trace Data

InvariantsInvariants

Invariants

Invariant Updates

(https/ssl)

Community Machine

Server Machine

What Does the Local Daikon Do?

• Local Daikon• Reads trace data • Performs invariant inference

• Standard set of invariants

• One of (var = one of {val1, …, valn})

• Not null (var != null)

• Less than (var1 - var2 < c)

• Many more (75 different kinds)• Variables from same basic block (for now)

Invariant Learning Mode Architecture

Tracing

Client Library

Determina MPEE

Application

Local Daikon

NodeManager

Central Daikon

ManagementConsole

InvariantDatabase

Trace Data

InvariantsInvariants

Invariants

Invariant Updates

(https/ssl)

Community Machine

Server Machine

What Does Central Daikon Do?

• Takes invariants from Local Daikons• Logically merges invariants into Invariant

Database• Each kind of invariant has merge rules• For example

•x = 5 merge x = 6 is x one-of {5, 6}•x > 0 merge x > 10 is x > 10•x = 5 merge no invariant about x is

no invariant about x•x = 5 merge no data yet about x is x

= 5

Application Community Issues

• Lots of community members learning at same time

• Each community member instruments a (randomly chosen) subset of basic blocks• Minimizes learning overhead• While obtaining reasonable coverage

• Learning takes place over successful executions (without attacks)• Controlled environment• A posteriori judgement

Monitoring Mode Architecture

Client Library

Determina MPEE

Application

NodeManager

Protection Manager

ManagementConsole

Attack Informatio

nAttack

Information

(https/ssl)

Community Machine

Server Machine

Attack Detection

Community Machine

• Detects attack signal• Determina Memory Firewall• Fatal error (invalid address, divide by

zero)• In principle, any indication of attack

• Attack information• Program counter where attack

occurred• Stack when attack occurred

• Sent to server as application dies

Invariant Localization Overview

• Goal: Find out which invariants are violated when program is attacked

• Strategy: • Find invariants close to attack • Make running applications check for

violations of these invariants• Correlate invariant violations with

attacks

Invariant Localization Mode Architecture

Attack & InvariantViolation Detector

Client Library

Determina MPEE

Application

NodeManager

Protection Manager

ManagementConsole

InvariantDatabase

Attack & Invariant

Information

Attack & Invariant

Information

Invariants

(https/ssl)

Community Machine

Server Machine

LiveShieldGeneration

LiveShieldInstallation

LiveShields

LiveShields LiveShield

s

Finding Invariants Close to Attack

• Attack Information• PC of instruction where attack detected

(jump to invalid code) (instruction that accessed invalid memory) (divide by zero instruction)

• Call stack•Duplicate stack•Preserved even for stack smashing

attacks• Find basic blocks that are close to involved

PCs• Find invariants for those basic blocks

Detecting Invariant Violations

• Add checking code to application• Check for violations of selected

invariants• Log any violated invariants

• Use Determina LiveShield mechanism• Distribute code patches to basic blocks• Eject basic blocks from code cache• Insert new version of basic block with

new checking code• Updates programs as they run

Using LiveShield Mechanism

• Protection manager selects invariants to check

• Generates C code that implements check• Passes C code to scripts

• Compile the code• Generate patch• Sign it, convert to LiveShield format

• Distribute LiveShields back to applications• Each application gets all LiveShields• Goal is to maximize checking information

Correlating Invariant Violations and Attacks

• Protection manager fed two kinds of information• Invariant violation information• Attack information

• Correlates the information• If invariant violation is followed by an

attack• Then invariant is a candidate for

enforcement

Protection Mode Architecture

Client Library

Determina MPEE

Application

NodeManager

Protection Manager

ManagementConsole

InvariantDatabase

Attack & Invariant

Information

Attack & Invariant

Information

Invariants

(https/ssl)

Community Machine

Server Machine

LiveShieldGeneration

LiveShieldInstallation

LiveShields

LiveShields LiveShield

s

Attack Detector & Invariant Enforcement

Invariant Enforcement

• Given an invariant to enforce• Protection manager generates LiveShields

that correspond to different repair options• Current implementation for one-of

constraints• Variable is a pointer to a function• Constraint violation is a jump to function

previously unseen at that jump instruction• Potential repairs

•Call one of previously seen functions•Skip call•Return immediately back to caller

Selecting A Good Repair

• Protection manager generates a LiveShield for each repair option

• Distributes LiveShields across applications• Random assignment, biased as follows• Each LiveShield has a success number

• Invariant enforcement followed by continued successful execution increments number

• Attack or crash decrements number• Probability of selection is proportional to

success number• Periodically reassign LiveShields to

applications

System in Action - Learning

Community Machines

Invariants

InvariantDatabase

Protection Manager

ManagementConsole

Server Machine

Invariants

System in Action - Monitoring

Community Machines

InvariantDatabase

Protection Manager

ManagementConsole

Server Machine

System in Action - Monitoring

Community Machines

InvariantDatabase

Protection Manager

ManagementConsole

Server Machine

Attack Informatio

n

System in Action – Invariant Localization

Community Machines

InvariantDatabase

Protection Manager

ManagementConsole

Server Machine

Invariants

Invariant Checks in LiveShield

s

System in Action – Invariant Localization

Community Machines

InvariantDatabase

Protection Manager

ManagementConsole

Server Machine

Invariant Violation Information

Attack Information

System in Action – Protection

Community Machines

InvariantDatabase

Protection Manager

ManagementConsole

Server Machine

Repair Distribution

System in Action – Protection

Community Machines

InvariantDatabase

Protection Manager

ManagementConsole

Server Machine

Invariant Violation Information

Attack Information

System in Action – Protection

Community Machines

InvariantDatabase

Protection Manager

ManagementConsole

Server Machine

Repair Redistributio

n

System in Action – Protection

Community Machines

InvariantDatabase

Protection Manager

ManagementConsole

Server Machine

Repair Redistributio

n

System in Action – Protection

Community Machines

InvariantDatabase

Protection Manager

ManagementConsole

Server Machine

System in Action – Protection

Community Machines

InvariantDatabase

Protection Manager

ManagementConsole

Server Machine

System in Action – Concrete Example

• Learning mode• Key binary variable is target of jsri instruction• Learn a one-of constraint

(target is one-of invoked functions) • Monitoring mode

• Memory Firewall detects attempt to execute unauthorized function

• Invariant localization mode• Attack information identifies jsri instruction

as target of attack• Correlates invariant violation with attack

• Protection Mode• Distribute range of repairs

(skip call, call previously observed function)• Check that they successfully neutralize attack

Attack Surface Issues

• Determina Runtime as attack target• Addressed with page protection policies

• Also randomize placement• Runtime data• Runtime code, code cache

Page Type Runtime Mode Application Mode

App code R R

App data RW RW

Runtime code RE R

Code Cache RW RE

Runtime data RW R

Communication Issues

• What about forged communications?• Management console has certificate

authority• Clients use password to get

certificates• All communications

•Signed, authenticated, encrypted•Revocation if necessary

InvariantDatabase

ManagementConsole

CertificateAuthority

Status

• Architecture implemented and tested• Components exist• Communication implemented,

operational• Determina Memory Firewall as attack

detector• One-of invariants on function pointers

(demo)

Parameterized Architecture and Implementation

• Parameterization points• Attack signal• Invariants

•Inference•Enforcement mechanisms

• Flexibility in implementation strategies• Invariant localization strategies• Invariant repair strategies

Class of Attacks

Prerequisites for stopping an attack• Attack characteristics

• Attack signal• Attack must violate invariants• Enforcing invariants must neutralize

attack• Invariant characteristics

• Daikon must recognize invariants• System must be able to successfully

repair violations of invariants

Examples of Attacks We Can Stop

• Function pointer • Attack signal – Determina Memory

Firewall• Invariant

•One-of invariant•Function pointer binary variable

• Repair•Jump to previously seen function•Skip call

Examples of Attacks We Can Stop

• Code injection attacks via stack overwriting• Attack signal – Determina Memory

Firewall• Invariant

•Less than invariant•Stack pointer binary variable

• Repair•Skip writes via binary variable•Coerce binary variable back into range

Future Evolution

• Exploit parameterization capabilities• More sophisticated invariants

• Data structure inference• Sequences of program actions

• More sophisticated repairs• More sophisticated attack signals

• Detect more subtle attacks•Program keeps executing•Executes legitimate code only

• Use invariant violation as attack signal