1. CS 589 Information Risk Management 23 January 2007

2. Today’s Discussion Start with risk Discuss types of information risk Start with systematic, modeling-based framework for assessing alternatives when risks are known Continue with the hard part – specification of risk when risks are unknown

3. Next Week Discuss specification of risks using probability distributions Discuss incorporation of this information into a decision tree Discuss ways to apply these techniques to Information Risk scenarios

4. After Next Week Discuss the Expected Utility decision criterion Discuss Multiple Objectives and Expected Value and Expected Utility Discuss Applications in Information Risk Analysis and Management

5. References for Today Clemen, R. L. and T. Reilly, Making Hard Decisions. Duxbury, 2001. Gaffney Jr., J. E., J. W. Ulvila, “Evaluation of Intrusion Detectors: A Decision Theory Approach”, Proceedings of the IEEE Symposium on Security and Privacy. 2001.

6. Risk ??? Chance of something bad happening? Having something bad happen? Anything else?

7. Risk The probability of an event occurring combined with the consequences of that event Just about everything is risky How do we actually measure risk?

8. Risk vs Uncertainty Uncertainty –We don’t know what the key variables are –We don’t know how they relate to alternatives Risk –Specify probability distributions –Connect them with alternatives One goal: Uncertainty  Risk via Modeling

9. Thinking About Risk Probabilities and Outcomes Which is riskier? –Living near a large power generation station –International flight –Driving to Albuquerque We have to define factors, events, outcomes, and associated probabilities

10. Dealing with Risk Define Risk Assess Risk Define Alternatives for Handling the Risk Evaluate Alternatives Evaluate your Evaluation Model Sensitivity Analysis Implementation

11. Evaluation Choosing among Alternatives Should be Evaluated on the same dimension(s) –Expected Value –Expected Utility –Value at Risk (VAR) –Multiple criteria Measurement of Alternatives on criteria dimensions is key – and another modeling issue

12. Sensitivity Analysis Checking on the evaluation of each alternative by varying individual variables Find the variable(s) that have the largest impact(s) on the ordering of alternatives Goal: robust solutions

13. Visual Representation Influence Diagrams –Connect factors, events –Help us define risks –Decomposition Decision Trees –Ordering of decisions, risky events –Easy to see and present – and solve

14. Visual Representations Squares denote Decisions Circles denote Risks Influence Diagrams – arcs connect decision and risk (aka chance) nodes Decision Trees – decision and chance nodes are sequentially ordered from left to right

15. A Very Simple Example Coin Flip Game Decisions: Play/No Play Risks: Heads/Tails Outcomes Must be Specified

16. Coin Flip Game Decision Tree With $0 Outcomes

17. If All Outcomes are $0 We are Indifferent between Play and No Play based on the Expected Value criterion We Prefer Play to No Play if E(Play) > E(No Play) Which means that the sum of the outcomes (if we have a fair coin) must be positive Generally, Play if

18. What if we can play twice? Sequential decision – we see the result of the first coin flip, and decide to continue This leads to the notion of Strategies – we can make a plan contingent upon resolution of risks that are resolved between decision nodes Everything is still based on Expected Value

20. Suppose O(H) = $10, O(T) = -$7 p(H) = p(T) =.5 (Fair coin) We can easily see that we would choose to Play in the one-game case What about the 2-game case?

22. Strategy It’s pretty simple – keep playing Would you really do this? Do you believe this? Why or why not??

23. Simple Example Suppose we are assessing two alternative intrusion detection systems. What’s the problem? What are the key risks for this decision? What are the decisions? What are the outcomes? How would we measure the outcomes? What is the decision criterion?

24. Key Point The optimal choice will be the one that is associated with the best expected criterion value – such as expected total cost This will be determined by how we define the outcomes – in terms of total costs – and probabilities When we roll back a decision tree, we assume that the downstream decision is the best one

25. Expected Value Random Variable with possible discrete outcomes

30. What do we need to know? Probabilities –P(Detection|An Intrusion)  P(D|I) –Associated Info –P(I) –And, finally, P(I|D) Outcomes –Individually, these will not be stochastic – for now –They will still lead to an expectation for each decision node

31. Conditional Probability P(D|I) and P(D| Not I) P(Not D|I) and P(Not D|Not I) Where would we get this information? What about P(I)?

32. Bayes Rule – Simple Version

33. Interpretation Two types of Accuracy Two types of Error

34. Solving the Tree Establish the Outcomes Compute the Probabilities – the conditionals on the endpoints and others Find Expected Values and roll back the tree

36. Sensitivity Analysis What are the strategies given the numbers we used in the example? What are the key variables? How would we assess the base-case outcome of this example?

37. Different Conditional Information What if we don’t know P(D|I)? We can flip the tree according to what we do know Outcomes should remain the same And the decision should remain the same

38. Another Way – Info Dependent

39. Modeling Decisions, chance events Probability distributions for chance events –Lack of data  Bayesian methods –Expert(s) –Lots of data  Distribution model(s) Outcomes –Financial, if possible –Multiple measures/criteria/attributes

40. Decision Situation In the context of Firm or Organization Goals, Objectives, Strategies A complete understanding should lead to a 1-2 sentence Problem Definition –Could be risk-centered –Could be oriented toward larger info issues Problem Definition should drive the selection of Alternatives and, to some degree, how they are evaluated

41. Information Business Issues Integrity and reliability of information stored and used in systems Preserve privacy and confidentiality Enhance availability of other information systems

42. Risk Management Process of defining and measuring or assessing risk and developing strategies to mitigate or minimize the risk Defining and assessing –Data driven –Other sources Developing strategies –Done in context of objectives, goals