Model-Checking In-lined Reference Monitors Meera Sridhar and Kevin W. Hamlen The University of Texas at Dallas January 17, 2010 Supported in part by grants

Model-Checking In-lined Reference Monitors

Meera Sridhar and Kevin W. HamlenThe University of Texas at Dallas

January 17, 2010

Supported in part by grants from AFOSR

VMCAI 2010Madrid, Spain

Sridhar and Hamlen: Model-Checking In-lined Reference Monitors

2

Reference Monitors

1/17/2010

OS/VM

REFERENCE MONITOR

grant/deny event

Examples: file system permissionsmemory safety

Disadvantages: • changing the policy requires changing the OS/VM• difficult to enforce finer-grained policies such as

“No modifications to files ending in .exe”

UNTRUSTED CODE


3

In-lined Reference Monitors [Schneider]

1/17/2010

OS/VM

REFERENCE MONITOR

grant/deny event

UNTRUSTED CODE

enforce safety policies by injecting runtime security guards directly into untrusted binarieso guards test whether the impending operation constitutes

a policy violation, and if so some corrective action is taken

maintain history of security-relevant events

Advantages:o No need to modify the OS/VM

o enforce richer policies: e.g., no network sends after file reads

o more flexible: code recipient can specify security policy

Examples: SASI [Erlingsson, Schneider], Java-MAC [Kim et al], Java-MOP [Chen, Rosu], Polymer [Bauer, Ligatti, Walker], ConSpec [Aktug, Naliuka], MoBILe [Hamlen, Morrisett, Schneider]


4

IRM Certification Problem

1/17/2010

REWRITER

UNTRUSTED CODE POLICY

VERIFIER reject

(rewriter failure)execute

REWRITTEN CODE

Main Points:

(a) TCB reduction(b) verifier must be

light-weight (e.g. suitable for hand-held devices)

Related work:ConSpec [Aktug, Naliuka] (certification via contracts), MoBILe [Hamlen et al] (certification via type-checking)

Trusted Computing Base


5

Overview of Contributions

• design and implementation of prototype IRM model-checking framework for ActionScript bytecode– ActionScript/AIR ubiquitous, existing security mechanisms center

around code-signing/sandboxing which are inadequate

• novel approach for constructing a state abstraction lattice from a security automaton– precise yet tractable abstract interpretation of IRM code

• rigorous proofs of soundness and convergence • feasibility shown by enforcing a URL anti-redirection policy for

ActionScript bytecode programs

1/17/2010


6

ActionScript IRM Examplepublic class RUnsafeURL extends Sprite {

// constructor for popup linkpublic function RUnsafeURL(){

var theField:TextField = new TextField(); theField.text = "Click me!"; addChild(theField); theField.addEventListener(MouseEvent.CLICK, clickListener);

}

// respond to user clicks by popping up infinite numbers of windowsprivate function clickListener(e:MouseEvent):void {

var url:String="javascript:window.open('http://popup.url.com');void(0);";

while(true) {

navigateToURL(new URLRequest("http://popup.url.com"));

} }

}

1/17/2010


7

ActionScript IRM Examplepublic class RUnsafeURL extends Sprite {

private var security:Number = 0;

// constructor for popup linkpublic function RUnsafeURL(){

var theField:TextField = new TextField(); theField.text = "Click me!"; addChild(theField); theField.addEventListener(MouseEvent.CLICK, clickListener);

}

// respond to user clicks by popping up infinite numbers of windowsprivate function clickListener(e:MouseEvent):void {

var url:String="javascript:window.open('http://popup.url.com');void(0);";

while(true) {if (security > 3) HALT;navigateToURL(new URLRequest("http://popup.url.com"));security++;

} }

}

1/17/2010

Policy: At most 3 pop-ups


8

Reified Security State• added by IRM (not in original program)• abstract the event history at some point in program’s execution

– Example: event counter• two operations: set and test

– both operations are statically identifiable– write-safety achieved via type-safety and object encapsulation (e.g.,

private fields)– verified by standard ActionScript bytecode verifier (similar to Java)– dangerous operations conservatively rejected (e.g., untrackable aliasing)

• values not necessarily current– stored state may reflect a past security state– Example: exhibit event, then increment counter later

1/17/2010


9

Synchronizing Reified State Variables

1/17/2010

TRACE BEFORE

REIFIED STATEBEFORE

CODE LINE

τ = ε s = 0 event eτ = e s = 0 s := 1τ = e s = 1 event eτ = ee s = 1 if s =1 then

…else …

• in practice, reified state value temporarily deviates from actual trace• unavoidable in practice, must be supported

•Solution: represent abstract state as a pair:(S, τ ‘),

abstract state when s was last updated

literal event sequence subsequent to last s update (usually ε or some very short sequence)


10

Verifying IRMs with Model-Checking

• introduce concrete trace meta-variable τ– tracks history of security-relevant events– implicitly modified by security-relevant operations

• verification strategy: abstract interpret τ-enhanced program to statically verify τ P⊆– deciding τ P is the model-checking step⊆– implemented as simple regular language subset in our

prototype, but more sophisticated models possible• main challenge: abstracting τ– too concrete: too slow to converge– too abstract: too much conservative rejection

1/17/2010


11

Abstract State Lattice

1/17/2010

q1

q2

q0

q3

e4

e0e2

e1

e3

T

Res(q0) Res(q1)

…

…

...

Res({q0,q1})

SECURITY AUTOMATON

STATE LATTICE

ordered by subset relation

we assume automata are deterministic


12

Security Policy

Definition 1 (Security Policy)

Let A = (Q,Σ, q0, δ) be a deterministic security automaton.

The security policy PA for automaton A is

PA= ResA(Q)

ResA(Q) = ∪q Q∈ ResA(q) where ResA(q) is the residual of state q in A.

1/17/2010


13

Abstract Traces

Definition 2 (Abstract Traces):

The language SS of abstract traces is SS = {(Res(Q0), τ) | Q0 Q, ⊆ τ Σ∈ ∗, |τ| ≤ k} {∪ ⊤SS }

where ⊤SS = Σ∗.

Abstract traces are ordered by subset relation , forming the lattice (SS, ).⊆ ⊆

1/17/2010


14

A Simple Example

1/17/2010

Security Policy : ε + e


15

A Simple Example, contd.

1/17/2010

Security Policy : ε + e


16

Abstract Machine

1/17/2010

LANGUAGE SYNTAX

ABSTRACT CONFIGURATIONS


17

Abstract Small-step Operational Semantics

1/17/2010


18

Soundness & Convergence• Proof of Soundness

– concrete machine• defines semantics of ActionScript VM• security violations modeled as stuck states

– abstract machine• defines semantics of abstract interpreter• rejection modeled as stuck state

– Progress Lemma: if the abstract machine makes progress, so does the concrete machine

– Preservation Lemma: whenever both machines make progress, the abstract machine abstracts the concrete machine

• Proof of Convergence– abstract machine reaches a fixed point within O(n2) iterations in the worst

case, where n is the size of the security automaton– proof bounds the height of the abstract state lattice

1/17/2010


19

Example Security Policy

1/17/2010

Policy Goal:

prohibit malicious URL-redirections by ABC ad applets

URL navigation in ActionScript Bytecode:

― redirections are implemented at the bytecode level by navigateToURL system calls

― our policy requires that method check_url(s) be called to validate s before any redirection to s

― check_url has a trusted implementation provided by ad distributor and/or web host, and may incorporate dynamic information) such as ad hit counts or webpage context.


20

Example Security Policy (contd.)

1/17/2010

Policy Enforcement:

― our IRM enforces this policy by injecting calls to check_url into untrusted applets

― for better runtime efficiency, it positions some of these calls early in the program's execution (to pre-validate certain URL's) and injects runtime security state variables that avoid potentially expensive duplicate calls by tracking the history of past calls


21

Example Security Policy (contd.)

• we used Prolog to build our IRM system– declarative nature of Prolog predicates yields clean, concise code– reversible nature of Prolog predicates helps reuse parsing code for

code-generation

• some preliminary numbers:

1/17/2010

PROGRAM TESTED SIZE BEFORE

SIZE AFTER

REWRITINGTIME

VERIFICATIONTIME

countdownBadge.abc 1.80 KB 1.95 KB 1.429s 0.532s

NavToURL.abc 0.93 KB 1.03 KB 0.863s 0.233s


22

Future Work

• extend work to support – reified security state per-object– reified security state updated by IRM before actual

security state changes at runtime

• support for recursion and concurrency• extend work to other binary languages and IRM

systems that have been implemented for them

1/17/2010


23

Selected Citations1. P. Cousot and R. Cousot. Abstract Interpretation: A unified lattice model for static analysis of

programs by construction or approximation of fixpoints. In Proc. Symposium on Principles of Prog. Languages, pages 234–252, 1977.

2. P. Cousot and R. Cousot. Abstract Interpretation Frameworks. J. Log. Comput., 2(4):511–547, 1992.

3. B. W. DeVries, G. Gupta, K. W. Hamlen, S. Moore, and M. Sridhar. ActionScript Bytecode Verification with Co-logic Programming. In Proc. of the ACM SIGPLAN Workshop on Prog. Languages and Analysis for Security (PLAS), 2009.

4. K. W. Hamlen, G. Morrisett, and F. B. Schneider. Certified In-lined Reference Monitoring on .NET. In Proc. ACM Workshop on Prog. Languages and Analysis for Security (PLAS), 2006.

5. K. W. Hamlen, G. Morrisett, and F. B. Schneider. Computability Classes for Enforcement Mechanisms. In ACM Trans. Prog. Languages and Systems, 2006.

6. F. B. Schneider. Enforceable Security Policies. ACM Trans. Information and System Security, 3:30–50, 2000.

7. M. Sridhar and K.W. Hamlen. ActionScript In-Lined Reference Monitoring in Prolog. In Proc. Intl. Symposium on Practical Aspects of Declarative Languages, 2010.

1/17/2010

Documents

Model-Checking In-lined Reference Monitors Meera Sridhar and Kevin W. Hamlen The University of Texas at Dallas January 17, 2010 Supported in part by grants