Automated Compliance Checking...Policies (sample) 1.Access-control rules restrict who accesses and...

Automated Compliance Checking

Felix Klaedtke

Institute of Information Security, ETH Zurich

Oxford, INQUEST 2012

1

Joint work with . . .

David Basin Matus Harvan Srdjan Marinovic

Samuel Muller Eugen Zalinescu2

Modern problems of Alice and Bob

financialinformation

calendardocumentsemails. . .social

networks

medicaldata

intellectual property

conflicts ofinterests

corporategovernanceregulatory

compliance

privacypolicies

How are these problems related?How do we address Alice’s and Bob’s concerns?

3

Modern problems of Alice and Bob

financialinformation

calendardocumentsemails. . .social

networks

medicaldata

intellectual property

conflicts ofinterests

corporategovernanceregulatory

compliance

privacypolicies

How are these problems related?How do we address Alice’s and Bob’s concerns?

3

At the core

� Controlling access“My medical data should only be accessible to my care givers”

� Controlling usage“. . . and used for intended purpose, e.g., improving my healthcare.”

� Implement controls to reduce risks

� For IT systems:processes that monitor and control how other processes access anduse data

Challenges: generality, scalability, automation, . . .

4

At the core

� Controlling access“My medical data should only be accessible to my care givers”

� Controlling usage“. . . and used for intended purpose, e.g., improving my healthcare.”

� Implement controls to reduce risks

� For IT systems:processes that monitor and control how other processes access anduse dataChallenges: generality, scalability, automation, . . .

4

Focus

policy

� Target: security and compliance

∗∗∗ Security policies, business processes∗∗∗ Policies regulating data usage and agent behavior∗∗∗ HIPAA, SOX, separation of duty, etc.

� Focus: monitoring

� Note: monitoring is different from enforcement

5

Focus

events

ComplianceChecker

during runtime or audit

6

� Target: security and compliance

∗∗∗ Security policies, business processes∗∗∗ Policies regulating data usage and agent behavior∗∗∗ HIPAA, SOX, separation of duty, etc.

� Focus: monitoring

� Note: monitoring is different from enforcement

5

Nokia’s data-collection campaign

� Phone data collected and propagated to databases:location, call and SMS info, accelerometer, . . .

� Participants can view and delete their data

� Clear-text data used for personalized apps, e.g., location-history maps

� Anonymized data is used for research6

Policies (sample)

1. Access-control rules restrict who accesses and modifies data in databases

(A) Only user script2 may delete data from db2

(B) Databases db1 and db2 are accessed by script1 account only while script1is running

2. Data changes are propagated between databases

(C) Data deleted from db2 is deleted from db3 within 60 seconds

(D) Data inserted into db1 is, within 30 hours, either inserted into db2 ordeleted from db1

7

Results of case study

� Performance:∗∗∗ One year of logged data: 220 million log entries (8GB)

policy time / space

easiest 17 minutes / 14 MB

hardest 1 hour / 3.3 GB (mostly within 600 MB)

∗∗∗ Processing times reasonable and space requirements manageable

� Compliance:∗∗∗ System users attempted unauthorized actions

∗∗∗ Testing, debugging, and other improvement activities

∗∗∗ Bugs in scripts and triggers

� Value:∗∗∗ Useful even in a benevolent environment where the enterprise is

committed to policy compliance

∗∗∗ Helpful to debug and sharpen controls

∗∗∗ Can be used to support audits, both internal and external8

Reminder of the talk

1.) Policy specification

2.) Monitoring algorithm

3.) Case study revisited

4.) Conclusion

9

Reminder of the talk

1.) Policy specification

2.) Monitoring algorithm

3.) Case study revisited

4.) Conclusion

9

Policy elements

1. Reports must be approved before they are published.

2. Approvals must happen at most 10 days before publication.

3. The employees’ managers must approve the reports.

10

Policy elements

1. Reports must be approved before they are published.

2. Approvals must happen at most 10 days before publication.

3. The employees’ managers must approve the reports.

Subjects� reports and employees

� unbounded over time

qqqqqqq qqqqqqqq qqqqqqqqq qqqqqqqqq qqqqqqqqqq qqqqqqqqqqq qqqqqqqqqqqq qqqqqqqqqqqqq qqqqqqqqqqqqqq qqqqqqqqqqqq qqqqqqqqqqqqq qqqqqqqqqqqqq qqqqqqqqqqqqq qqqqqqqqqqqqq qqqqqqqqqqqqqq qqqqqq qqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqq qqqqqqq qqqqqqqq

r rrrrrrrrrrr rrrrrrrrrrr rrrrrrrrrrr rrrrrrrrrrrr rrrrrrrrrr rrrrrrrrrr rrrrrrrrr rrrrrrrrr rrrrrrrr rrrrrrrr

10

Policy elements

1. Reports must be approved before they are published.

2. Approvals must happen at most 10 days before publication.

3. The employees’ managers must approve the reports.

Subjects� reports and employees

� unbounded over time

Temporal aspects� qualitative: before and always

� quantitative: at most 10 days

qqqqqq qqqqqqq qqqqqqqq qqqqqqqqq qqqqqqqqqq qqqqqqqqqqq qqqqqqqqqqqq qqqqqqqqqqqqqq qqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqq qqqqqqq qqqqqq qqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqq qqqqqqq qqqqqqq

rrrrrrrrr rrrr rrrrr rrrrr rrrrrr rrrrrrrr rrrrrrrrrrrr rrrrrrrrrrrr rrrrrrrrrrrrr rrrrrrrrrrrrr rrrrrrrrrrrrrr rrrrrrrrrrrrrrr

10

Policy elements

1. Reports must be approved before they are published.

2. Approvals must happen at most 10 days before publication.

3. The employees’ managers must approve the reports.

Subjects� reports and employees

� unbounded over time

Temporal aspects� qualitative: before and always

� quantitative: at most 10 days

Event predicates� approving and publishing a report

� happen at a point in time

� logged with time-stamp

qqqqqqqq qqqqqqqq qqqqqqqqq qqqqqqqqqq qqqqqqqqqqq qqqqqqqqqqqq qqqqqqqqqqqqq qqqqqqqqqqqqqq qqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqqq qqqqqqqqqqqqqqq qqqqqqqqqqqqqq qqqqqqqqqqqqq qqqqqqqq qqqqqqq qqqqqq qqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqqqqqqqqqqq qqqqqqqq qqqqqqqqq qqqqqqqqqq

r rrrrrrrrrrr rrrrrrrrrr rrrrrrrr rrrrrrr rrrrrr rrrrrr r rrrrrrrrrrrrr rrrrrrrrrrrr rrrrrrrrrrrr rrrrrrrrrrrrr

10

Policy elements

1. Reports must be approved before they are published.

2. Approvals must happen at most 10 days before publication.

3. The employees’ managers must approve the reports.

Subjects� reports and employees

� unbounded over time

Temporal aspects� qualitative: before and always

� quantitative: at most 10 days

Event predicates� approving and publishing a report

� happen at a point in time

� logged with time-stamp

State predicates� being someone’s manager

� have a duration

qqqqqq qqqqqqq qqqqqqqqq qqqqqqqqqq qqqqqqqqqqq qqqqqqqqqqqq qqqqqqqqqqqqq qqqqqqqqqqqqqq qqqqqqqqqqqqqqq qqqqqqqqqqqqqqqqq qqqqqqqqqqq qqqqqqqqqq qqqqqqqqq qqqqqqqqq qqqqqqqq qqqqqqq qqqqqq qqqqqqq qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq qqqqqq qqqqqq qqqqqqq

r rrrrrrrr rrrrrrrrr rrrrrrrrrr rrrrrrrrrrr rrrrrrrrrrrr rrrrrrrrrrrrr rrrrrrrrrrrrrr rrrrrrrrrrrrrrr rrrrrrrrrrrrrrrr rrrrrrrrrrrrrrrrr

10

Specification languageMetric First-Order Temporal Logic (MFOTL)

���∀e.∀r . publish report(e, r)→�≤10 ∃m.

((¬managerfinish(m, e)) S managerstart(m, e)

)∧

approve report(m, r)

� First-order for expressing relations on data

� Temporal operators for reasoning about time

��� ! “at every future time-point” (always)

� ! “at some time-point in the past” (once)

� Metric information adds timing constraints

�≤10 ψ-τ0 τ1 τ2 τ3 τ4 τ5

ψ︸ ︷︷ ︸τ4−τ1≤10 11

Syntax

� A signature S is a tuple (C ,R)

∗∗∗ C is a finite set of constant symbols∗∗∗ R is a finite set of predicates, each with an associated arity

� (MFOTL) formulas over signature S and set of variables V :

φ ::= t1 ≈ t2∣∣ r(t1, . . . , tn)

∣∣ ¬φ ∣∣ φ ∧ φ ∣∣ ∃x . φ∣∣

Iφ∣∣###Iφ

∣∣ φ SI φ∣∣ φUI φ ,

where ti range over V ∪ C , r over R, x over V , and I is an interval of the

form [b, b′) = {a ∈ N | b ≤ a < b′}, for b ∈ N, b′ ∈ N ∪ {∞}, and b < b′

� Syntactic sugar: �I φ = ¬(true SI ¬φ), ���I φ = ¬(true UI ¬φ), . . . ,

���<3 φ = ���[0,3) φ, . . . , and non-metric operators like ���φ = ���[0,∞) φ

12

Semantics (I)

-

time-pointstime-stamps

first-order structures

0τ0

D0

1τ1

D1

2τ2

D2

3τ3

D3

. . .

. . .

. . .

� A temporal (first-order) structure (over S) is a pair (D, τ)

∗∗∗ Sequence τ = (τ0, τ1, . . . ) of time-stamps, τi ∈ NMonotonically increasing and progressing

∗∗∗ Sequence of first-order structures D = (D0,D1, . . . ).

Constant domains and rigid interpretation of constant symbols

� (D, τ , v , i) |= φ denotes MFOTL satisfaction

(D, τ) is a temporal structure, v a valuation, i ∈ N, and φ a formula

� Standard semantics for first-order constructs

13

Semantics (II)Metric temporal operators

A temporal formula is only satisfied at time-point i if it is satisfiedwithin the bounds given by interval I , relative to time-stamp τi

(D, τ , v , i) |= I φ iff i > 0, τi − τi−1 ∈ I , and (D, τ , v , i − 1) |= φ

(D, τ , v , i) |=###I φ iff τi+1 − τi ∈ I and (D, τ , v , i + 1) |= φ

(D, τ , v , i) |= φ SI ψ iff for some j ≤ i , τi − τj ∈ I , (D, τ , v , j) |= ψ,and (D, τ , v , k) |= φ, for all k ∈ {j + 1, . . . , i}

(D, τ , v , i) |= φUI ψ iff for some j ≥ i , τj − τi ∈ I , (D, τ , v , j) |= ψ,and (D, τ , v , k) |= φ, for all k ∈ {i , . . . , j − 1}

Example

φ S[3,17) ψ -0τ0

1τ1

2τ2

3τ3

4τ4

5τ5

ψ φ φ φ︸ ︷︷ ︸3≤ τ4−τ1 < 17

14

Remarks

� Why temporal operators?

∗∗∗ Separates temporal aspects from data aspects∗∗∗ Restricts temporal reasoning to a few cases

� Why a discrete time domain?

∗∗∗ Clocks have limited precision∗∗∗ Minor impact on monitoring

� Why time-points with time-stamps?

∗∗∗ Event-based view∗∗∗ Temporal reasoning with points is “simpler” than with intervals

� Experience: MFOTL is well suited to express a wide range of policies

15

Reminder of the talk

1.) Policy specification

2.) Monitoring algorithm

3.) Case study revisited

4.) Conclusion

16

Monitoring objective

� For a policy given as an MFOTL formula φ

���∀c .∀t. ∀a. trans(c , t, a) ∧ th < a→ ���<6 report(t)

� and a prefix of a temporal structure given by system events or logs

- timeτ0

trans cID tID amountBob #34 $100’000Eve #37 $1’000

report tID

τ1

trans cID tID amountEve #45 $999’999

report tID#34

. . . τi

trans cID tID amountBob #78 $24Mallory #99 $333’333

report tID

. . .

� monitor should report all policy violations (either online or offline)

17

Monitoring objective

� For a policy given as an MFOTL formula φ

��� ∀c .∀t.∀a. trans(c , t, a) ∧(�<31 ∃t ′.∃a′. t 6≈ t ′ ∧ trans(c , t ′, a′) ∧ ���<6 report(t ′)

)→���<3 report(t)

� and a prefix of a temporal structure given by system events or logs

- timeτ0

trans cID tID amountBob #34 $100’000Eve #37 $1’000

report tID

τ1

trans cID tID amountEve #45 $999’999

report tID#34

. . . τi

trans cID tID amountBob #78 $24Mallory #99 $333’333

report tID

. . .

� monitor should report all policy violations (either online or offline)

17

Monitoring objective

� For a policy given as an MFOTL formula φ

��� ∀c .∀t.∀a. trans(c , t, a) ∧(�<31 ∃t ′.∃a′. t 6≈ t ′ ∧ trans(c , t ′, a′) ∧ ���<6 report(t ′)

)→���<3 report(t)

� and a prefix of a temporal structure given by system events or logs

- timeτ0

trans cID tID amountBob #34 $100’000Eve #37 $1’000

report tID

τ1

trans cID tID amountEve #45 $999’999

report tID#34

. . . τi

trans cID tID amountBob #78 $24Mallory #99 $333’333

report tID

. . .

� monitor should report all policy violations (either online or offline)

17

Restrictions-

τ0

D0

τ1

D1

τ2

D2

τ3

D3 . . .

. . . ?

6|= φ

Not all MFOTL-definable properties and temporal structures can be effectivelymonitored

� Formula φ of form ���φ′, where φ′ is bounded

∗∗∗ For all occurrences of operator U[b,b′) in φ′, b′ 6=∞∗∗∗ So φ describes a safety property

� Structures D0,D1, . . .

∗∗∗ Option 1. AutomaticRoughly, each structure Di representable by a collection of finite automata,

see, e.g., [Khoussainov & Nerode ’95] and [Blumensath & Gradel ’04]

∗∗∗ Option 2. Finite relations (a special case of option 1)But then negation and quantification must be appropriately handled, e.g.,

r(x) ∧ ���I ¬q(x) ; r(x) ∧ ¬ ���I q(x)

We have taken Option 2 in our implementation (more details later)

18

Preprocessing: negation and rewriting

� Input formula φ

���∀t.∀c .∀a. trans(t, c , a) ∧(�<31 ∃t ′.∃a′. t 6≈ t ′ ∧ trans(t ′, c , a′) ∧ ���<6 report(t ′)

)→���<3 report(t)

� Negate, rewrite, and drop outermost ��� and ∃ quantifier(s), yielding ψ

��� ∃t. ∃c .∃a. trans(t, c , a) ∧(�<31 ∃t ′.∃a′. t 6≈ t ′ ∧ trans(t ′, c , a′) ∧ ���<6 report(t ′)

)∧¬ ���<3 report(t)

PPP���

� For monitoring: for each i ∈ N, determine elements satisfying ψ:{a∣∣ (D, τ , v [x/a], i) |= ψ

}These are the transactions that should have been reported at time-point i

19

Preprocessing: negation and rewriting

� Input formula φ

���∀t.∀c .∀a. trans(t, c , a) ∧(�<31 ∃t ′.∃a′. t 6≈ t ′ ∧ trans(t ′, c , a′) ∧ ���<6 report(t ′)

)→���<3 report(t)

� Negate, rewrite, and drop outermost ��� and ∃ quantifier(s), yielding ψ

��� ∃t. ∃c .∃a. trans(t, c , a) ∧(�<31 ∃t ′.∃a′. t 6≈ t ′ ∧ trans(t ′, c , a′) ∧ ���<6 report(t ′)

)∧¬ ���<3 report(t)

PPP���

� For monitoring: for each i ∈ N, determine elements satisfying ψ:{a∣∣ (D, τ , v [x/a], i) |= ψ

}These are the transactions that should have been reported at time-point i

19

Preprocessing: negation and rewriting

� Input formula φ

���∀t.∀c .∀a. trans(t, c , a) ∧(�<31 ∃t ′.∃a′. t 6≈ t ′ ∧ trans(t ′, c , a′) ∧ ���<6 report(t ′)

)→���<3 report(t)

� Negate, rewrite, and drop outermost ��� and ∃ quantifier(s), yielding ψ

��� ∃t. ∃c .∃a. trans(t, c , a) ∧(�<31 ∃t ′.∃a′. t 6≈ t ′ ∧ trans(t ′, c , a′) ∧ ���<6 report(t ′)

)∧¬ ���<3 report(t)

PPP���

� For monitoring: for each i ∈ N, determine elements satisfying ψ:{a∣∣ (D, τ , v [x/a], i) |= ψ

}These are the transactions that should have been reported at time-point i

19

Preprocessing: reduction to first-order queries

� For each temporal subformula α in ψ, introduce an auxiliary predicate pα

∃c .∃a. trans(t, c , a) ∧(�<31 ∃t ′.∃a′. . . . ∧ ���<6 report(t ′)︸ ︷︷ ︸

pα1︸ ︷︷ ︸pα2

)∧ ¬ ���<3 report(t)︸ ︷︷ ︸

pα3

� Replace each α by a corresponding pα, yielding first-order formula ψ

∃c .∃a. trans(t, c , a) ∧ pα2(c , t) ∧ ¬pα3(t)

� For monitoring:

∗∗∗ For each i ∈ N, extend Di to Di , where for each temporal subformula α

pDiα =

{a∣∣ (D, τ , v [x/a], i) |= α

}∗∗∗ For each i ∈ N, query extended first-order structure Di{

a∣∣ (Di , v [x/a]) |= ψ

}Next: how to construct pDi

α for each i ∈ N

20

Preprocessing: reduction to first-order queries

� For each temporal subformula α in ψ, introduce an auxiliary predicate pα

∃c .∃a. trans(t, c , a) ∧(�<31 ∃t ′.∃a′. . . . ∧ ���<6 report(t ′)︸ ︷︷ ︸

pα1︸ ︷︷ ︸pα2

)∧ ¬ ���<3 report(t)︸ ︷︷ ︸

pα3

� Replace each α by a corresponding pα, yielding first-order formula ψ

∃c .∃a. trans(t, c , a) ∧ pα2(c , t) ∧ ¬pα3(t)

� For monitoring:

∗∗∗ For each i ∈ N, extend Di to Di , where for each temporal subformula α

pDiα =

{a∣∣ (D, τ , v [x/a], i) |= α

}∗∗∗ For each i ∈ N, query extended first-order structure Di{

a∣∣ (Di , v [x/a]) |= ψ

}Next: how to construct pDi

α for each i ∈ N

20

Preprocessing: reduction to first-order queries

� For each temporal subformula α in ψ, introduce an auxiliary predicate pα

∃c .∃a. trans(t, c , a) ∧(�<31 ∃t ′.∃a′. . . . ∧ ���<6 report(t ′)︸ ︷︷ ︸

pα1︸ ︷︷ ︸pα2

)∧ ¬ ���<3 report(t)︸ ︷︷ ︸

pα3

� Replace each α by a corresponding pα, yielding first-order formula ψ

∃c .∃a. trans(t, c , a) ∧ pα2(c , t) ∧ ¬pα3(t)

� For monitoring:

∗∗∗ For each i ∈ N, extend Di to Di , where for each temporal subformula α

pDiα =

{a∣∣ (D, τ , v [x/a], i) |= α

}∗∗∗ For each i ∈ N, query extended first-order structure Di{

a∣∣ (Di , v [x/a]) |= ψ

}

Next: how to construct pDiα for each i ∈ N

20

Preprocessing: reduction to first-order queries

� For each temporal subformula α in ψ, introduce an auxiliary predicate pα

∃c .∃a. trans(t, c , a) ∧(�<31 ∃t ′.∃a′. . . . ∧ ���<6 report(t ′)︸ ︷︷ ︸

pα1︸ ︷︷ ︸pα2

)∧ ¬ ���<3 report(t)︸ ︷︷ ︸

pα3

� Replace each α by a corresponding pα, yielding first-order formula ψ

∃c .∃a. trans(t, c , a) ∧ pα2(c , t) ∧ ¬pα3(t)

� For monitoring:

∗∗∗ For each i ∈ N, extend Di to Di , where for each temporal subformula α

pDiα =

{a∣∣ (D, τ , v [x/a], i) |= α

}∗∗∗ For each i ∈ N, query extended first-order structure Di{

a∣∣ (Di , v [x/a]) |= ψ

}Next: how to construct pDi

α for each i ∈ N20

Constructing the auxiliary relations

- timeτ0

D0 . . .

. . . τi−1

Di−1

τi

Di

Di

τi+1

Di+1 . . .

. . .

� Construct auxiliary relations pαDi inductively over α’s formula structure and using

also relations from both previous and subsequent structures

� Case where α has form I β: pαDi =

{βDi−1 if i > 0 and τi − τi−1 ∈ I

∅ otherwise

� Case where α has form ###I β: pαDi =

{βDi+1 if τi+1 − τi ∈ I

∅ otherwise

∗∗∗ Construction depends on relations in Di+1 for which the predicates occur in β

∗∗∗ Monitor constructs pαDi with a delay of at least one time step

21

Construction for S[0,∞)

� The construction for α = β S[0,∞) γ reflects the logical equivalence

α↔ γ ∨ (β ∧ α)

� Assume that β and γ have the same free variables. Then

pαDi = γDi ∪

{∅ if i = 0

βDi ∩ pαDi−1 if i > 0

� Uses relations just for subformulas and previous time-point

� Constructions for metric SI and UI slightly more involved

22

Monitoring algorithm

1: i ← 0 % lookahead index in sequence (D0, τ0), (D1, τ1), . . .2: q ← 0 % index of next query evaluation in sequence (D0, τ0), (D1, τ1), . . .3: Q ←

{(α, 0,waitfor(α)

) ∣∣α temporal subformula of ψ}

4: loop5: Carry over constants and relations of Di to Di .6: for all (α, j , ∅) ∈ Q do % can build relation for α in Dj

7: Build auxiliary relation for α in Dj .8: Discard auxiliary relation for α in Dj−1 if j − 1 ≥ 0.

9: Discard relations pDj

δ , where δ is a temporal subformula of α.

10: while all relations pDqα are built for α ∈ tsub(ψ) do

11: Output violations ψDq and time-stamp τq.12: Discard structure Dq−1 if q > 0.13: q ← q + 114: Q ←

{(α, i + 1,waitfor(α)

) ∣∣α temporal subformula of ψ}∪{(

α, j ,⋃α′∈update(S,τi+1−τi )

waitfor(α′)) ∣∣ (α, j , S) ∈ Q and S 6= ∅

}15: i ← i + 1 % process next element in input sequence (Di+1, τi+1)16: end loop

Counters q (query) and i (lookahead) into input sequence

23

Monitoring algorithm

1: i ← 0 % lookahead index in sequence (D0, τ0), (D1, τ1), . . .2: q ← 0 % index of next query evaluation in sequence (D0, τ0), (D1, τ1), . . .3: Q ←

{(α, 0,waitfor(α)

) ∣∣α temporal subformula of ψ}

4: loop5: Carry over constants and relations of Di to Di .6: for all (α, j , ∅) ∈ Q do % can build relation for α in Dj

7: Build auxiliary relation for α in Dj .8: Discard auxiliary relation for α in Dj−1 if j − 1 ≥ 0.

9: Discard relations pDj

δ , where δ is a temporal subformula of α.

10: while all relations pDqα are built for α ∈ tsub(ψ) do

11: Output violations ψDq and time-stamp τq.12: Discard structure Dq−1 if q > 0.13: q ← q + 114: Q ←

{(α, i + 1,waitfor(α)

) ∣∣α temporal subformula of ψ}∪{(

α, j ,⋃α′∈update(S,τi+1−τi )

waitfor(α′)) ∣∣ (α, j , S) ∈ Q and S 6= ∅

}15: i ← i + 1 % process next element in input sequence (Di+1, τi+1)16: end loop

Q maintains list of unevaluated subformula (α, j , S) for past time-points

23

Monitoring algorithm

1: i ← 0 % lookahead index in sequence (D0, τ0), (D1, τ1), . . .2: q ← 0 % index of next query evaluation in sequence (D0, τ0), (D1, τ1), . . .3: Q ←

{(α, 0,waitfor(α)

) ∣∣α temporal subformula of ψ}

4: loop5: Carry over constants and relations of Di to Di .6: for all (α, j , ∅) ∈ Q do % can build relation for α in Dj

7: Build auxiliary relation for α in Dj .8: Discard auxiliary relation for α in Dj−1 if j − 1 ≥ 0.

9: Discard relations pDj

δ , where δ is a temporal subformula of α.

10: while relations pDqα are built for all temporal subformulas α of ψ do

11: Output violations ψDq and time-stamp τq.12: Discard structure Dq−1 if q > 0.13: q ← q + 114: Q ←

{(α, i + 1,waitfor(α)

) ∣∣α temporal subformula of ψ}∪{(

α, j ,⋃α′∈update(S,τi+1−τi )

waitfor(α′)) ∣∣ (α, j , S) ∈ Q and S 6= ∅

}15: i ← i + 1 % process next element in input sequence (Di+1, τi+1)16: end loop

Given relations for all temporal subformulas, output policy violations

23

Finite relations

� In each iteration, monitor stores auxiliary relations

� Problem: must restrict negation and quantification

∗∗∗ Consider the formula p(x) ∧ ¬q(x)

∗∗∗ In (i + 1)st iteration, monitor constructs auxiliary relation pDi

¬q(x)

� Solution: rewrite to a formula so that auxiliary relations are finite

∗∗∗ p(x) ∧ ¬q(x) is rewritten to p(x) ∧ (¬q(x) ∧### p(x)

)∗∗∗ Heuristic!

∗∗∗ Related to domain independence of database queries, e.g., [Fagin ’82]

� Under reasonable assumptions, the size of the finite relations ispolynomially bounded w.r.t. to input

24

Finite relations

� In each iteration, monitor stores auxiliary relations

� Problem: must restrict negation and quantification

∗∗∗ Consider the formula p(x) ∧ ¬q(x)

∗∗∗ In (i + 1)st iteration, monitor constructs auxiliary relation pDi

¬q(x)

� Solution: rewrite to a formula so that auxiliary relations are finite

∗∗∗ p(x) ∧ ¬q(x) is rewritten to p(x) ∧ (¬q(x) ∧### p(x)

)∗∗∗ Heuristic!

∗∗∗ Related to domain independence of database queries, e.g., [Fagin ’82]

� Under reasonable assumptions, the size of the finite relations ispolynomially bounded w.r.t. to input

24

� Implementation of the monitoring algorithm

∗∗∗ Input: monpoly -sig signature -formula policy -log logfile∗∗∗ Output: policy violations

� Open source, GNU public license

� Written in OCaml

� Download it at http://projects.developer.nokia.com/MonPoly

� Best Tool Award at conference on Runtime Verification (2011)

25

Performance evaluation

� Generated log files with different event rates for a fixed time span� Monitoring performance for complex transaction-report policy:

0

100

200

300

400

500

600

0 50 100 150 200 250 300

seco

nd

s

time units

2k eventstime unit

10

15

20

25

30

35

40

45

0 50 100 150 200 250 300

MB

time units

2k eventstime unit

0

200

400

600

800

1000

1200

1400

0 500 1000 1500 2000 2500 3000 3500 4000

seco

nd

s

event rates

•

10

15

20

25

30

35

40

45

50

55

0 500 1000 1500 2000 2500 3000 3500 4000

MB

event rates

•

� PostgreSQL does not scale to larger log files26

Reminder of the talk

1.) Policy specification

2.) Monitoring algorithm

3.) Case study revisited

4.) Conclusion

27

Logging

����

����

����� ��

� ��

� Log entries are produced atmultiple places

� Need to merge logs

� No total order on log entries

� Compliance might depend on orderIn general, an intractable problem

Log sample@unix time-stampevent db user db data id

@1272902328insert (eu.030, db1, 146368038)insert (eu.031, db2, 122368122)@1272902355delete (script2, db2, 108031209)select (res.012, db3, 146368038)@1273158243script end (script1)

28

Intractability

� Instead of monitoring a single trace, we must monitor a set of traces

� Policy violation: some trace/all traces

� Even for a very restrictive setting, corresponding decision problemsare intractable

Instance:

∗∗∗ propositional, past-only, non-metric linear-time temporal formula φ

∗∗∗ prefixes D1 and D2 of length n ≥ 1with D i = (D i

1, τ1) (D i1, τ1) . . . (D i

n, τn), for i ∈ {1, 2}Question WEAK: (D, 2n) 6|= φ, for some D ∈ D1 |||||| D2 is NP-complete

Question STRONG: (D, 2n) 6|= φ, for all D ∈ D1 |||||| D2 is coNP-complete

29

Pragmatic solution

� Policies should not care about the ordering of events with equaltime-stamps

���∀u.∀d . delete(u, db2, d)→ �<1s ���<60s ∃u′. delete(u′, db3, d)

� Even more drastic: monitor the log stream in which all events withequal time-stamps are merged

log stream 10 3

3

7 7

3

99

0 7 9

14

14 14

14

log stream 2

merged and collapsed

log stream

� Checking if an MFOTL formula is order-independent is undecidable∗∗∗ However, inductively reasoning over formula structure is often sufficient

∗∗∗ Reasoning can be automated with proof rules

t ≈ t′ : (|=∀) t ≈ t′ : ( 6|=∀)

ψ : (|=∃)

�I ���J ψ : (|=∀)0 ∈ I ∩ J

30

Pragmatic solution

� Policies should not care about the ordering of events with equaltime-stamps

���∀u.∀d . delete(u, db2, d)→ �<1s ���<60s ∃u′. delete(u′, db3, d)

� Even more drastic: monitor the log stream in which all events withequal time-stamps are merged

log stream 10 3

3

7 7

3

99

0 7 9

14

14 14

14

log stream 2

merged and collapsed

log stream

� Checking if an MFOTL formula is order-independent is undecidable∗∗∗ However, inductively reasoning over formula structure is often sufficient

∗∗∗ Reasoning can be automated with proof rules

t ≈ t′ : (|=∀) t ≈ t′ : ( 6|=∀)

ψ : (|=∃)

�I ���J ψ : (|=∀)0 ∈ I ∩ J

30

Reminder of the talk

1.) Policy specification

2.) Monitoring algorithm

3.) Case study revisited

4.) Conclusion

31

Current and future directions ?!

� Scaling upHow do we monitor billions instead of millions of log entries?How can we effectively monitor distributed systems distributively?

∗∗∗ Algorithmic improvements∗∗∗ Trace slicing and parallelization

� Incomplete knowledgeHow account for actions that are not logged (e.g, logging failures)?What if observations are contradictory?

∗∗∗ Multi-valued logics, e.g., Belnap (t, f,⊥,>)∗∗∗ Probabilities

� EnforcementWhat policies are enforceable (in a concurrent setting)?How does an enforcement mechanism interact with a system?Can we synthesize enforcement mechanisms from policies?

32

Conclusion

� Policy monitoring is a challenging and increasingly relevant topic!

� Logical methods are well suited for reasoning about policies

Instance MFOTL: expressive, yet monitoring practically feasible

� Tool support publicly availableat http://projects.developer.nokia.com/MonPoly

� No silver bullet

∗∗∗ Not every policy can be formalized in MFOTL

∗∗∗ Running times and space consumption is still (always!) an issue

33

References

1. D. Basin, F. Klaedtke, S. Muller, and B. Pfitzmann. Runtimemonitoring of metric first-order temporal properties. FSTTCS’08

2. D. Basin, F. Klaedtke, and S. Muller. Monitoring security policieswith metric first-order temporal logic. SACMAT’10

3. D. Basin, F. Klaedtke, and S. Muller. Policy monitoring in first-ordertemporal logic. CAV’10

4. D. Basin, M. Harvan, F. Klaedtke, and E. Zalinescu. Monitoringusage-control policies in distributed systems. TIME’11

5. D. Basin, F. Klaedtke, and E. Zalinescu. Algorithms for monitoringreal-time properties. RV’11

6. D. Basin, M. Harvan, F. Klaedtke, and E. Zalinescu. MONPOLY:Monitoring usage-control policies. RV’11

7. D. Basin, V. Juge, F. Klaedtke, and E. Zalinescu. Enforceablesecurity policies revisited. POST’12

8. D. Basin, F. Klaedtke, S. Marinovic and E. Zalinescu. Monitoringcompliance policies over incomplete and disagreeing logs. RV’12

34