Preventing Information Leaks with Jeeves · Jeeves Supports Expressive Policies def isNotCarol(oc): return oc != Jean Yang / Jeeves 20 Output context can be of arbitrary type. def

Preventing

Information Leaks

with Jeeves

Jean Yang, MITApril 8, 2015

Wearable

devices

Data, Data Everywhere

Jean Yang / Jeeves 2

Social

media

Electronic health

records

Online courses


All Kinds of People Are

Writing All Kinds of

CodeOpen source

lines of code

Journalists

Medical

researchers

Social

scientists

Children


Even Trained

Developers Leak

Information

Why Aren’t Existing

Approaches Enough?


Exploit

Patch

But leaves system

builders a step

behind.

Defensive

protection

But people are still

showing the data

wrong.

Encrypting

Data

My Approach:

Privacy by Construction


Factor out privacy to reduce

opportunity for leaks.

• Programmer specifies high-level policies about

how sensitive data can be used.

• Rest of program is policy-agnostic.

• System manages policies automatically.

Social Calendar Example


Alice and Bob throw a surprise party for Carol.

Even Seemingly Simple

Policies Have Subtleties


Guests Carol Strangers

Surprise

party for

Carol at

Chuck E.

Cheese.

Pizza with

Alice/Bob.

Private event

at Chuck E.

Cheese.

Policy: Must be guest. Policies can depend on

sensitive values and other

policies.Policy: Only visible to hosts until finalized.

Problem:

Enforcing Policies Can

Leak Information!


Guests

Surprise

party at

Chuck E.

Cheese.

Policy: Only visible to

hosts until finalized.

Policy: Must be guest.

Guest list finalized

Guests can’t see

event

Guests can see

event

• Subtle mistake:

check for policy 1

neglects

dependency on

policy 2.

• Problem arises

when programmers

trusted to get

dependencies right.

1

2

Policies Are Intertwined

Across the Code


“What is the most

popular location

among friends 7pm

Tuesday?”

Update to

event

subscribers

• Track information flow through derived values.

• Track where derived values flow.

Problem:


“Policy Spaghetti”

in Real Systems

Code from

HotCRP

conference

management

system

Highlighted: conditional permissions checks everywhere.


Programming

model.

Mathematical

guarantees.

Implementations,

web framework,

and case studies.

Automatic Enforcement

with JeevesThe well-intentioned programmer

writes same code no matter what

policies are.

Jeeves Factors Out

Policies


• Centralized policies.

• Policy-agnostic program.

• Runtime differentiates behavior.

Model View Controller

Background.


Jeeves programming model.

How Jeeves works.

Theoretical guarantees.

Implementation & evaluation.

This Talk

[PLAS ’13]

[PLAS ’13]

[draft]

[POPL ’12]

Background.

1. Specifying policies.

2. Specifying differentiated behavior.

3. Factoring out policies from other

code.

4. Enforcing policies that depend on

sensitive values.

Automatically Managing

Information Flow


Only Jeeves supports 3 and 4.

Jeeves supports more

expressive policies

than prior work.


Information flow controls check that only allowed flows occur [HiStar,

Flume, Resin, LIO/Hails, Jif/Sif, flow locks, F*, and more].

Related Work:

Checking Information

Flow

==

true

==

false

if viewer == : elif viewer != :

error

Programmer

tags data.

System prevents

leaks by raising

error.

Must explicitly

define and tag

alternate behavior.

My previous work.


==

Multi-execution approaches differentiate behavior based on viewer

[secure multi-execution (Devriese et al); faceted execution (Austin and

Flanagan)].

true false

Computing with Multiple

Values

Related Work:

Must encode

policies as tags.• “Policy

spaghetti.”

• Policies can

leak

information.

Background.


Jeeves programming model

How Jeeves works.



This Talk

Jeeves programming model.

Supporting

expressive policies

that can depend on

sensitive values.

2

Factoring out

information flow.

1

Challenges

Policy-Agnostic

Programming in Jeeves


Application CodeSeparate from policies.

policies

Sensitive values

encapsulate multiple

behaviors.

Policies describe rules for

how values may flow to

output contexts.

.guests [ ]

Jeeves Supports

Expressive Policies

def isNotCarol(oc): return oc !=


Output context can be of arbitrary type.

def isGuest(oc): return oc in .guests

A policy is an arbitrary function that takes the

output context and returns a Boolean value.

Policies can depend on sensitive values.

21Jean Yang / Jeeves

==

true false

print { } print { }

true false

Jeeves Programming

ModelProgrammer

writes policy-

agnostic

programs.

Runtime

propagates values

and policies.

Runtime

produces

differentiated

output based on

the viewer.

Programmer

specifies policies

and facets.

1 2

4

3

The Jeeves

Programming Model• Well-defined runtime semantics

for policy-agnostic programming with information flow policies.

• Can be implemented standaloneor embedded as a library.

• Has been adapted across runtimes in web frameworks.


Background.



How Jeeves works.



This Talk

How Jeeves works.

Handling policies

with dependencies.

1

Enforcing

policies when

state changes.

2

Challenges

24Jean Yang / Jeeves

if == :

x += 1

return x

x = 0

print { } print { }

1 0

Jeeves Execution Model

Runtime

propagates

values and

policies.

Runtime

solves for

values to show

based on

policies and

viewer.

21

Runtime simulates simultaneous multiple

executions.

Using Policies to

Produce Outputs


print { }

0

( != ) ? 1 : 0

policy( )

Jeeves uses

policies to defacet

appropriately.

1 0

def isNotCarol(oc):

return oc !=


print { }

( == ) ? :

policy( )

def isMaybeCarol(oc):

return oc ==

But What About

Dependencies?

Possible solutions:

( == ) ? :

( == ) ? :

Jeeves runtime

will pick the secret

value if allowed.

Need to find a

fixed point!


Using Constraints to

Handle DependenciesLabel Policy

a def isGuest(oc):

return oc in .guests

𝑎 = 𝑠𝑒𝑐𝑟𝑒𝑡 ⇒ in .guests

policy( )

𝑎 ∈ {𝑠𝑒𝑐𝑟𝑒𝑡, 𝑝𝑢𝑏𝑙𝑖𝑐}

print { }

𝑎 = 𝑠𝑒𝑐𝑟𝑒𝑡 ⇒ 𝑓𝑎𝑙𝑠𝑒

¬(𝑎 = 𝑠𝑒𝑐𝑟𝑒𝑡)

⊢

⊢

0

1 0

a

• Constraints contain

only Boolean

variables.

• Always a consistent

assignment.

Evaluated with

respect to state

at time of

output.

Handling Indirect Flows


if == :

a

x += 1

true false

aif :

x += 1

x = xold+1 xold

a

Labels follow

values through all

computations,

including

conditionals and

assignments.

Outputting to Sinks


Σ, 𝐸𝑜𝑐 ⇓∅ Σ𝑜𝑐 , 𝑉𝑜𝑐Σ𝑜𝑐 , 𝐸𝑟 ⇓∅ Σ𝑟 , 𝑉𝑟

𝑘1…𝑘𝑛 = 𝑐𝑙𝑜𝑠𝑒𝐾(𝑙𝑎𝑏𝑒𝑙𝑠 𝐸𝑜𝑐 ∪ 𝑙𝑎𝑏𝑒𝑙𝑠(𝐸𝑟), Σ2)

𝐸𝑝 = 𝜆𝑥. 𝑡𝑟𝑢𝑒 ∧𝑓 …∧𝑓 Σ2(𝑘𝑛)

Σ𝑟 , (𝐸𝑝 𝑉𝑜𝑐) ⇓∅ Σ𝑝, 𝑉𝑝

pick 𝑝𝑐 such that 𝑝𝑐 𝑉𝑜𝑐 = 𝑜𝑐, 𝑝𝑐 𝑉𝑟 = 𝑅, 𝑝𝑐 𝑉𝑝 = 𝑡𝑟𝑢𝑒

Σ, print 𝐸𝑜𝑐 𝐸𝑟 ⇓ 𝑉𝑝, 𝑜𝑐: 𝑅

Evaluate output context

and expression to print.

Retrieve labels

and policies.Evaluate policies applied

to the output context.

Defacet using

satisfying policy

assignment.

Jeeves Runtime Semantics:

Σ, 𝑆 ⇓ 𝑉𝑝, 𝑜𝑐: 𝑅 Σ, 𝐸 ⇓𝑝𝑐 Σ′, 𝑉

Statement

evaluation

Expression

evaluation

Output

context

ResultPolicies

Background.



How Jeeves works.



This Talk


Proving

enforcement with

respect to our

semantics.

2

Defining what it

means to enforce

policies.

1

Challenges

Non-Interference


a

if == :

x += 1

print { }

0

0 1

a

aif == :

x += 1

Jeeves Challenge:

Compute labels

from program—may

have dependencies

on secret values!

Secret values should not affect public output.


a

if == :

x += 1

print { }

0

0 1

a

aif == :

x += 1

Jeeves Theorem:

All executions where

a must be public

produce equivalent

outputs.

Jeeves Non-Interference

and Policy Compliance

Can’t tell apart secret values

that require a to be public.

𝑓: 𝑅 ∃𝑉. ∅, 𝑆1 ⇓ 𝑉, 𝑓: 𝑅} = {𝑓: 𝑅 | ∃𝑉. ∅, 𝑆2 ⇓ 𝑉, 𝑓: 𝑅}

𝑆2 = 𝑝𝑟𝑖𝑛𝑡 𝑒 𝐶[ ]𝑆1 = 𝑝𝑟𝑖𝑛𝑡 𝑒 𝐶[ ]

Jeeves Policy

Compliance TheoremSuppose we have


𝐸1 𝐸𝑃𝑎

Then the set of values 𝑆1 evaluates to is equivalent to the set of values 𝑆2 evaluates to.

Where 𝑎 must be public.

𝐸2 𝐸𝑃𝑎

Need to account for policies that depend on sensitive values!

Background.



How Jeeves works.



This Talk

Implementation and evaluation.

Adapting the

approach for web

applications.

1

Scaling the

approach.

2

Challenges

Facets in the Application

and Database


ApplicationQueries

select * from Users

where location =

SQL

Database

Application All data SQL

Databaseselect * from Users

Database

queries can

leak

information!

Impractical

and

potentially

slow!

Solution: Facet the database. Implemented as an object-

relational mapping supporting uniform facet representation.

Jeeves runtime

Jacqueline Web

Framework


Application

Frontend Database

@jeevesViewerAttach policies.

Programmer is responsible

Framework is responsible


Django-like

data schema

for describing

fields.

Policy for

‘location’ field.

Helper

functions for

policy include

queries.

Public value for

‘location’ field.


Conference

management system

Course manager Medical records

(HIPAA)

Implemented in Jeeves

Questions1.How does faceted execution scale?

2.How does faceting the database affect

queries?

Demo


CMS Running Times


Database size does not affect

query speed.

Showing sensitive data grows

linearly.

0

0.05

0.1

0.15

0.2

0.25

0 200 400Tim

e to load p

age (

s)

Number of papers in DB

Show Single Paper

512 users 1024 users

0

10

20

30

40

50

0 200 400Tim

e to load p

age(s

)

Number of papers in DB

Show All Papers

512 users 1024 users

Tests from Amazon AWS machine via HTTP requests from another machine.

Evaluation Lessons

• Faceted execution is expensive.

• Policy-agnostic programming

does not always need faceted

execution.

• Many opportunities for

optimization in web frameworks.


Policy-Agnostic

Programming in Jeeves


Design of a policy-

agnostic

programming

language

[POPL ‘12]

Semantics and

guarantees

[PLAS ’13]

Implementation,

web framework,

and case studies

[in submission]

==

Other functionality

PoliciesSensitive

values


F*: type-checking

security properties.

[ICFP ’11; JFP ‘13]

Ask Reeves: verified

privacy policies.

[patent filed]

Verified

Type-

checked

Verve: a provably

type-safe OS.

[PLDI ’10 (Best Paper);

CACM ‘10]

My Interests in

Provable Guarantees

Lessons

• Difficult to write secure and private programs.

• Even more difficult to prove them correct.

• Easier if we can get privacy by construction.


The Future of Policy-

Agnostic Programming

• Factor out policies from other code.

• Rely on compiler and runtime to

differentiate behavior.

• Programmability.– Languages for different kinds of policies.

– Tools for verifying and visualizing policies.

• Scaling.– Policy-agnostic semantics with minimal faceting.

– Building foundations with policy management in mind.


Making Policy-Agnostic

Programming Work

Web

frameworks

Databases Web browsers

Parting Thoughts

By reducing opportunity for

programmer error, we can

eliminate whole classes of

privacy leaks.


Documents

Preventing Information Leaks with Jeeves · Jeeves Supports Expressive Policies def isNotCarol(oc): return oc != Jean Yang / Jeeves 20 Output context can be of arbitrary type. def