1. Advertisement 1

2. Audit Mechanisms for Provable Risk Management and Accountable Data Governance Jeremiah Blocki, Nicolas Christin, Anupam Datta, Arunesh Sinha Carnegie Mellon University 2

3. Motivation 3 Goal: treatment Rigid access control hinders treatment Permissive access control ⇒ privacy violations Breach

4. A real problem 4

5. Auditing 5  Audit – instead of rigid access control  Have a permissive access control regime  Inspect accesses later to find violations  Punish violators  Repetitive process  Audits - Why Cry Over Spilt Milk?  deters (near) rational employees

6. Audit Challenges 6  How much and what to audit?  Within budgetary constraints  How much to punish?  Without de-motivating employees  Human in the loop  Realistic model of human behavior

7. Contribution 7  A formal repeated game model of the audit process  An asymmetric equilibrium concept for games  An audit mechanism that is an equilibrium  Demonstrate usefulness of the model and equilibrium  Predicts commonly observed phenomenon  Predicts interesting results that calls for empirical analysis “essentially, all models are wrong, but some are useful” - George Box

8. Outline 8  Game Model  Equilibrium concepts  Equilibrium of Audit game  Predictions  Budget allocation and Fairness 1 2 3 4 5

9. Repeated Game Model 9  The interaction repeats for each audit cycle (rounds of repeated game)  Typical actions in one round  Emp action: (a, v) = (30, 2)  Org action: ( α, P) = (0.33, $100) Inspect Access, Violate Punishment rate One audit cycle (round) 1 Game Model J. Blocki, N. Christin, A. Datta, A. Sinha, Regret Minimizing Audits: A Learning-Theoretic Basis for Privacy Protection, IEEE Computer Security Foundations, 2011

10. Abstractions 10  Independence assumptions  K types of violations (and accesses)  Each employee acts independently for each type  One repeated game for each type and employee  Parameters of the model known through studies[P][V]  Risk factors (cost of violations)  Audit cost  Employee benefit in violating  ….  Infinite horizon audit interaction for fixed parameters [Game Theory, Fudenberg and Tirole] 1 Game Model [P] Ponemon Institute Studies, [V}Verizon Data Breach Studies

11. Violation detection 11  Given v violations and α fraction inspection  Expected number of violations caught internally - v. f( α )  Violations caught externally  Assume fixed probability p of external detection  Expected number – p.v.(1 – f( α )) 1 Game Model

12. Reputation Loss Audit Cost High Punishment Rate Loss Payoffs 12  Organization’s payoff  Employee’s payoff 1 Game Model ∝ α.a ∝ P ∝ p.v.(1 – f ( α )) ∝ v.f ( α ) Personal Benefit Punishment PB.v P.v.(p.(1 – f ( α )) + f ( α ))

13. Additional Considerations 13  Employees likely to not act rationally  Computationally constrained, Wrong beliefs  ϵ probability of arbitrary behavior  Org’s expected payoff for fixed P, α and employee action (a,v)  (1 - ϵ ).(expected payoff with (a,v)) + ϵ.(expected payoff with (a,a)) 1 Game Model Worst Case

14. Graphical View of Payoffs 14  Different employee best response partitions organization’s action space  Best response: v = 0 in deterred, v = a in un-deterred  More generally with non-linear payoff, a best response of k number of violations defines a partition 1 Game Model Fraction of accesses inspected ( α ) Punishment Rate (P) Deterred Un-Deterred PB α P 0 1 3 2a

15. Subgame Perfect Equilibrium 15  Strategy σ: nodes → actions  Pay( σ1,σ2) = δ -discounted sum of round payoffs  ( σ1,σ2) is NE if no unilateral profitable deviation  Node N defines a subgame G N with restricted strategy σ1 N  (σ1,σ2) is SPE if (σ1 N,σ2 N ) is NE for G N 2 Equilibrium concepts {} aa’ab’ba’bb’ ab’; aa’ Action of P1 = {a, b} Action of P2 = {a,’ b’}

16. Asymmetric approximate equilibrium 16  Any SPE has the single stage deviation property  Pay( σ1 sd,σ2) ≤ Pay( σ1,σ2)  Pay( σ1,σ2 sd ) ≤ Pay( σ1,σ2)  ϵ -SPE allows ϵ deviation by either player  ( ϵ 1, ϵ 2)-SPE allows ϵ 1, ϵ 2 deviation by player P1, player P2  Special relevant case for security: ( ϵ 1, 0)-SPE  Attacker (player P2) has no incentive to deviate  Deviations by attacker may be costly for defender 2 Equilibrium concepts

17. Proposed equilibrium 17  Organization: maximize utility subject to best response of employee (Stackelberg games)  Commitment by organization  Employee plays best response 3 Equilibrium The equilibrium attained is an ( ϵ 1, 0) SPE α P Deterred Un-Deterred PB ϵ 1 is the sum of a) difference from optimum due to uncertainty in PB b) ϵ. maximum loss in reputation

18. Advantages of commitment 18  Makes the decision easier for not so rational employee  Computing single round best response is easier  Predictable employee response – not based on beliefs (beliefs affected by many factors)  Addresses the problem of equilibrium selection  “Open design: The design should not be secret”[SS] 3 Equilibrium [SS] The Protection of Information in Computer Systems, Saltzer, J. H. and Schroeder, M. D.

19.  Doctors punished less than nurses  Punishing a doctor is more costly for hospitals  Less audit cost, better tools means more inspections  Organizations audit to protect against greater loss  Increasing difference in cost of externally and internally caught violation leads to more inspections  Should be studied empirically  Can be used as an effective policy tool  Data Breach Notiifcation law [SR] vs. External audits Predictions 19 4 Predictions [SR]Romanosky, S., Hoffman, D., Acquisti, A., Empirical analysis of data breach litigation, International Conference on Information Systems. (2011)

20. Budget Allocation 20  Organization plays multiple games  Organization is constrained by total budget  Let the games be 1….n. Let the budget be B.  Budget b i yields equilibrium Eq(b i ) in game i  Eq(b i ) results in payoff Pay(b i ) in game i  Solve max ∑ i Pay(b i ) subject to ∑ i b i ≤ B 5 Fair Auditing

21. Towards Accountable Data Governance 21  Utility maximization may lead to unfair allocation  Add fairness constraints  Minimum level of inspection, punishment rate for each type 5 Fair Auditing

22. Conclusion 22  Future Work:  Study the accountability problem in depth  Study complexity/algorithmic aspects of computing equilibrium Audit near-rational employees to optimize organization’s utility in a fair manner

23. References 23  Zhao, X., Johnson, M.E., Access governance: Flexibility with escalation and audit, Hawaii International International Conference on Systems Science, 2010  Zhang, N., Yu, W., Fu, X., Das, S.K.,Towards effective defense against insider attacks: The establishment of defender’s reputation, IEEE International Conference on Parallel and Distributed Systems. (2008)  Cheng, P.C., Rohatgi, P., Keser, C., Karger, P.A., Wagner, G.M., Reninger, A.S., Fuzzy Multi-Level Security : An Experiment on Quantified Risk-Adaptive Access Control, Proceedings of the IEEE Symposium on Security and Privacy. (2007)  Feigenbaum, J., Jaggard, A.D., Wright, R.N., Towards a formal model of accountability, Proceedings of the 2011 workshop on New security paradigms workshop. (2011)

24. 24