1. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 1 Unit Outline Quantitative Risk Analysis Module 1: Quantitative Risk Analysis and ALE Module 2: Case Study  Module 3: Cost Benefit Analysis & Regression Testing Module 4: Modeling Uncertainties Module 5: Summary

2. Module 3 Cost Benefit Analysis & Regression Testing

3. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 3 Students should be able to: –Understand how to use matrices for cost benefit analysis. –Calculate risk leverage. –Comprehend how regression testing is used. Cost Benefit Analysis Learning Objectives

4. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 4 Cost Benefit Analysis Matrix Cost Benefit Analysis The exposure before controls is equal to the summation of the aggregate values for impact value x threat value. (Vulnerability/Threat Matrix); In this case, the value is equal to: $1,617,234.13 The exposure after controls is equal to the sum of all of the multiplied threat importance values. For example, in the Hardware Failure column, we will take each of the threat importance values and subtract them each from 1. These values should be multiplied together. (Threat/Control Matrix) –This will give us: (1-.10) x (1 -.10) x (1 -.70) x (1 -.20) = 0.1944 –This value will be multiplied by the threat importance value: 0.1944 x $10907.90 = $2120.48 (cost with controls of Hardware Failure) –Do this for all threat columns and then summate all the values. This value is equal to: $33,780.67

5. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 5 We are using this equation to calculate cost: –C i = C si + C ri x t –Where C i is the total cost of control i. –C si is the static (one-time) cost of the control. –C ri is the additional cost per day (maintenance, updates, etc.) for the control. –t is equal to time (if calculating for a year, would equal 365). We show how to compute the costs of the controls for example cases. Spare Laptops: $2,500 x 200 = $500,000 –Warranties (3 year): $100 x 4,000 (laptops & desktops) + $1000 x 10 (regional servers) + $1,200 (HQ Server) = $411,200 –Physical Controls: $50,000 –Security Policy (creation, implementation, enforcement): $640 x 365 = $233,600 It is left to the user to accurate compute the cost of the controls and then compare the exposure with and without controls Cost Benefit Analysis Matrix Example

6. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 6 Costs are associated with both: –Potential Risk Impact –Reducing Risk Impact Risk Leverage is the difference in risk exposure divided by the cost of reducing the risk Let –r f be the risk exposure after imposing controls –r i be the risk exposure prior to imposing controls –c be the cost of controls Leverage l = (r i -r f )/c This tells you how many times the reduction in risk exposure is greater then the cost of controls. Cost Benefit Analysis Risk Leverage

7. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 7 Scenario: A company uses a common carrier to link to a network for certain computing applications. The company has identified the risks of unauthorized access to data and computing facilities through the network. These risks can be eliminated by replacement of remote network access with the requirement to access the system only from a machine operated on the company premises. The machine is not owned; a new one would have to be acquired. Cost Benefit Analysis Example #4: Unauthorized access

8. ItemAmount Risk: unauthorized access and use Access to unauthorized data and programs $100,000 @ 2% likelihood per year $2,000 Unauthorized use of computing facilities $10,000 @ 40% likelihood per year $4,000 Expected annual loss (2,000 + 4,000)$6,000 Effectiveness of network control: 100%-$6,000 Cost Benefit Analysis Example #4: Unauthorized Access Cost/Benefit Analysis for Replacing Network Access

9. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 9 Network Control cost: Hardware (50,000 amortized over 5 years)+$10,000 Software (20,000 amortized over 5 years)+$4,000 Support personnel (each year)+$40,000 Annual cost$54,000 Expected annual loss (6,000 – 6,000 +54,000)$54,000 Savings (6,000 – 54,000)-$48,000 Cost Benefit Analysis Example #4: Unauthorized Access

10. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 10 Scenario: This is a case where use of regression testing is being considered after making an upgrade to fix a security flaw. We want to determine if regression testing is economical in this scenario. Regression Testing means applying tests to verify that all remaining functions are unaffected by the change. Lets refer to the diagram on the following slide, to compare the risk impact of doing regression testing with not doing it. Upper part of the diagram –the risk of conducting regression testing Lower part of the diagram –shows the risks of not doing regression testing Regression Testing Example #5: Graphical Cost Benefit Analysis

11. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 11 In the two cases, one of three things can happen if regression is done: –We find a critical fault –We miss finding the critical fault –There are no critical faults to be found. For each possibility –Calculate the probability of an unwanted outcome, P(UO). –Associate a loss with that unwanted outcome, L(UO). Regression Testing Example #5: Cost Savings

12. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 12 Combined Risk Exposure Do regression testing? yes no P(UO) = 0.75 P(UO) = 0.05 P(UO) = 0.20 P(UO) = 0.05 P(UO) = 0.75 P(UO) = 0.20 Find critical fault Don’t find critical fault No critical fault Find critical fault Don’t find critical fault No critical fault L(UO) = $0.5M L(UO) = $30M L(UO) = $0.5M L(UO) = $30M L(UO) = $0.5M Risk Exposure $0.375M $1.500M $0.100M $1.975M $0.125M $16.500M $0.100M $16.725M Regression Testing Example #5: Calculation In our example, if we do regression testing and miss a critical fault in the system (a probability of 0.05), the loss could be $30 million. Multiplying the two, we find the risk exposure for that strategy to be $1.5 million. As the calculations in the figure prove, it is much safer to do regression testing than to skip it.

13. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 13 Do a cost benefit analysis based on the matrix that you have created for your own organization. Cost Benefit Analysis Assignment

14. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 14 Cost Benefit Analysis & Regression Testing Summary Cost Benefit Analysis is useful in determining whether the costs of controls is actually beneficial in terms of actual return or savings than the losses incurred by the risks they are meant to mitigate. Cost Benefit Analysis LEVERAGE = (RISK EXPOSURE before reduction – RISK EXPOSURE after reduction ) ________________________________________________ COST OF REDUCTION Regression Testing –Used for comparing risk impact

15. Sanjay Goel, School of Business/Center for Information Forensics and Assurance University at Albany Proprietary Information 15 Leverage l = (r i -r f )/c –r i = $251,037.60 x 365 = $91,628,724 –r f = $15,851.19 x 365 = $5,785,684.35 –C = $30,864,796 $251,037 – $15,851.19 / $30,864,796 =.008 $91,628,724 - $5,785,684.35 / $30,864,796 = 2.78 –The reduction in risk exposure is almost 3x greater than the cost of controls Cost Benefit Analysis Matrix Example