1

Advertisement

2

Audit Mechanisms for Provable Risk Management and Accountable Data

GovernanceJeremiah Blocki, Nicolas Christin, Anupam Datta, Arunesh Sinha

Carnegie Mellon University

3

Motivation

• Goal: treatment• Rigid access control hinders treatment• Permissive access control ⇒ privacy violations

Breach

4

A real problem

5

Auditing Audit – instead of rigid access control

Have a permissive access control regime Inspect accesses later to find violations Punish violators Repetitive process

Audits - Why Cry Over Spilt Milk? deters (near) rational employees

6

Audit Challenges How much and what to audit?

Within budgetary constraints

How much to punish? Without de-motivating employees

Human in the loop Realistic model of human behavior

7

Contribution A formal repeated game model of the audit process

An asymmetric equilibrium concept for games

An audit mechanism that is an equilibrium

Demonstrate usefulness of the model and equilibrium Predicts commonly observed phenomenon Predicts interesting results that calls for empirical

analysis

“essentially, all models are wrong, but some are useful” - George Box

8

Outline Game Model

Equilibrium concepts

Equilibrium of Audit game

Predictions

Budget allocation and Fairness

1

2

3

4

5

9

Repeated Game Model

The interaction repeats for each audit cycle (rounds of repeated game)

Typical actions in one round Emp action: (a, v) = (30, 2) Org action: (α, P) = (0.33, $100)

InspectAccess , Violate

Punishment rate

One auditcycle (round)

1 Game Model

J. Blocki, N. Christin, A. Datta, A. Sinha, Regret Minimizing Audits: A Learning-Theoretic Basis for Privacy Protection, IEEE Computer Security

Foundations, 2011

10

Abstractions Independence assumptions

K types of violations (and accesses) Each employee acts independently for each type One repeated game for each type and employee

Parameters of the model known through studies[P][V] Risk factors (cost of violations) Audit cost Employee benefit in violating ….

Infinite horizon audit interaction for fixed parameters [Game Theory, Fudenberg and Tirole]

1 Game Model

[P] Ponemon Institute Studies, [V}Verizon Data Breach Studies

11

Violation detection Given v violations and α fraction inspection

Expected number of violations caught internally - v. f(α)

Violations caught externally Assume fixed probability p of external detection Expected number – p.v.(1 – f(α))

1 Game Model

12

Reputation Loss

Audit Cost

High Punishment Rate Loss

Payoffs Organization’s payoff

Employee’s payoff

1 Game Model

∝ α.a

∝ P∝ p.v.(1 – f(α))∝ v.f(α)

PersonalBenefit PunishmentPB.v P.v.(p.(1 – f(α)) + f(α))

13

Additional Considerations Employees likely to not act rationally

Computationally constrained, Wrong beliefs ϵ probability of arbitrary behavior Org’s expected payoff for fixed P, α and employee

action (a,v) (1 - ϵ).(expected payoff with (a,v)) + ϵ.(expected payoff

with (a,a))

1 Game Model

Worst Case

14

Graphical View of Payoffs Different employee best response partitions

organization’s action space Best response: v = 0 in deterred, v = a in un-

deterred More generally with non-linear payoff, a best

response of k number of violations defines a partition

1 Game Model

Fraction of accesses inspected (α)

Punishment Rate (P)

Deterred

Un-Deterred PB

α

P

0 1

32 a

15

Subgame Perfect Equilibrium Strategy σ: nodes → actions Pay(σ1,σ2) = δ-discounted

sum of round payoffs (σ1,σ2) is NE if no unilateral

profitable deviation

Node N defines a subgame GN with restricted strategy σ1N

(σ1,σ2) is SPE if (σ1N,σ2N) is NE for GN

2 Equilibrium concepts

{}

aa’

ab’

ba’

bb’

ab’; aa’

Action of P1 = {a, b}Action of P2 = {a,’ b’}

16

Asymmetric approximate equilibrium Any SPE has the single stage deviation property

Pay(σ1sd,σ2) ≤ Pay(σ1,σ2) Pay(σ1,σ2sd) ≤ Pay(σ1,σ2)

ϵ-SPE allows ϵ deviation by either player (ϵ1, ϵ2)-SPE allows ϵ1, ϵ2 deviation by player

P1, player P2

Special relevant case for security: (ϵ1, 0)-SPE Attacker (player P2) has no incentive to deviate Deviations by attacker may be costly for defender

2 Equilibrium concepts

17

Proposed equilibrium Organization: maximize

utility subject to best response of employee (Stackelberg games)

Commitment by organization

Employee plays best response

3 Equilibrium

The equilibrium attained is an (ϵ1, 0) SPE

α

P Deterred

Un-Deterred PB

ϵ1 is the sum of a) difference from optimum due to uncertainty in PBb) ϵ . maximum loss in reputation

18

Advantages of commitment Makes the decision easier for not so rational

employee Computing single round best response is easier

Predictable employee response – not based on beliefs (beliefs affected by many factors)

Addresses the problem of equilibrium selection

“Open design: The design should not be secret”[SS]

3 Equilibrium

[SS] The Protection of Information in Computer Systems, Saltzer, J. H. and Schroeder, M. D.

19

Doctors punished less than nurses Punishing a doctor is more costly for hospitals

Less audit cost, better tools means more inspections

Organizations audit to protect against greater loss

Increasing difference in cost of externally and internally caught violation leads to more inspections Should be studied empirically Can be used as an effective policy tool

Data Breach Notiifcation law [SR] vs. External audits

Predictions4 Predictions

[SR]Romanosky, S., Hoffman, D., Acquisti, A., Empirical analysis of data breach litigation, International Conference on Information

Systems. (2011)

20

Budget Allocation Organization plays multiple games Organization is constrained by total budget

Let the games be 1….n. Let the budget be B. Budget bi yields equilibrium Eq(bi) in game i Eq(bi) results in payoff Pay(bi) in game i Solve max ∑i Pay(bi) subject to ∑i bi ≤ B

5 Fair Auditing

21

Towards Accountable Data Governance Utility maximization may lead to unfair

allocation

Add fairness constraints Minimum level of inspection, punishment rate for

each type

5 Fair Auditing

Money for celeb inspe...

Money for celeb inspec-tion

22

Conclusion

Future Work: Study the accountability problem in depth Study complexity/algorithmic aspects of

computing equilibrium

Audit near-rational employees to optimize organization’s utility in a

fair manner

23

References Zhao, X., Johnson, M.E., Access governance: Flexibility with

escalation and audit, Hawaii International International Conference on Systems Science, 2010

Zhang, N., Yu, W., Fu, X., Das, S.K.,Towards effective defense against insider attacks: The establishment of defender’s reputation, IEEE International Conference on Parallel and Distributed Systems. (2008)

Cheng, P.C., Rohatgi, P., Keser, C., Karger, P.A., Wagner, G.M., Reninger, A.S., Fuzzy Multi-Level Security : An Experiment on Quantified Risk-Adaptive Access Control, Proceedings of the IEEE Symposium on Security and Privacy. (2007)

Feigenbaum, J., Jaggard, A.D., Wright, R.N., Towards a formal model of accountability, Proceedings of the 2011 workshop on New security paradigms workshop. (2011)

24