1. A Framework for Comparing Different Information Security Risk Analysis Methodologies Anita Vorster, Les Labuschagne Ajay Kumar Iduri Sushma Sachidanand Quang Tran Vinh

2. Overview Introduction Information Security Risk Management Methodologies Criteria on which Framework is based The Framework for Comparison How the Framework should be used Strengths and Weakness of this proposed Framework Conclusion References

3. Introduction Information security is an organization’s approach to maintaining confidentiality, availability, integrity, nonrepudiation,accountability, authenticity and reliability of its IT systems Currently there are numerous risk analysis methodologies available some of which are qualitative while others are quantitative in nature These methodologies have a common goal of estimating the overall risk value

4. Introduction An easy-to-use framework is required to compare information security risk analysis methodologies The best way to choose between methodologies is to compare them, using objective, quantifiable criteria This is where a framework for comparison is needed If the criteria that are used are applicable to all risk analysis methodologies, the organization can compare different methodologies objectively, and decide on the best one

5. Alternative Frameworks The framework proposed by Badenhorst indicates whether a methodology addresses a criterion or not It does not use scales, or trade-offs which can aid the organization in choosing a methodology which will best meet their needs This shows the need for more Comparative Frameworks

6. Information Security Risk Management Methodologies The Methodologies can be broadly classified into two categories Quantitative Methodologies Qualitative Methodologies Both qualitative and quantitative methodologies were evaluated for developing the framework Different risk analysis methodologies were analyzed to determine common criteria for comparison

7. Qualitative Methodologies The qualitative methodologies considered for this framework are OCTAVE (Operationally Critical Threat, Asset and Vulnerability Evaluation) The CORAS (Construct a platform for Risk Analysis of Security Critical Systems) methodology

8. Quantitative Methodologies The quantitative methodologies considered for this framework are ISRAM (Information Security Risk Analysis Method) Cost-Of-Risk Analysis (CORA) Information Systems (IS) analysis based on a business model The above methodologies were chosen because they have been well documented

9. OCTAVE OCTAVE was developed at the CERT Coordination Center (CERT/CC) [Cert Coordination Center 2003] This approach concentrates on assets, threats and vulnerabilities One of the main concepts of OCTAVE is self- direction, the people inside the organization must lead the information security risk evaluation An analysis team, consisting of staff from the organization's business units as well as the IT department, is responsible for leading the evaluation and recording results

10. OCTAVE The OCTAVE approach has three phases, with each broken down into processes Each process has certain activities that must be completed, and within each of these activities, different steps must be taken in order to achieve the desired outputs The final result that risk decisions can be based on is the threat profile of different assets Each threat profile contains information on which mitigation decisions can be based

11. CORAS CORAS was developed under the Information Society Technologies (IST) program One of the main objectives of CORAS is to develop a framework that exploits methods for risk analysis, semi-formal methods for object-oriented modeling, and computerized tools, for a precise, unambiguous, and efficient risk assessment of security critical systems The methodology is based on UML a language that uses diagrams to illustrate relationships and dependencies between users and the environment in which they work

12. CORAS During an information security risk analysis, a great deal of information is brainstormed, and during workshops and discussions, different people (users, system developers, analysts, system managers), with different expertise in different fields come together, give their opinions and share information A way in which all the participants can communicate efficiently and understand each other must therefore exist and a UML profile, proposed by the CORAS project, is used to achieve this

13. CORAS The framework has four main pillars, of which risk management is one. In CORAS, the final result on which decisions can be based is the UML class diagrams of each asset

14. ISRAM The ISRAM methodology was developed at the National Research Institute of Electronics and Cryptology and the Gebze Institute of Technology in Turkey It is marketed as a quantitative approach to risk analysis that allows for the participation of the manager and staff of the organization and a survey-based model Two separate and independent surveys are conducted for the two attributes of risk, namely probability and consequence

15. ISRAM ISRAM does not use techniques such as Single Occurrence Losses (SOL) or Annual Loss Expectancy (ALE), instead, the risk factor is a numerical value between 1 and 25 This numerical value corresponds to a qualitative, high, medium or low value, and it is this qualitative value on which risk management decisions are based

16. CORA International Security Technology, Inc. (IST) developed CORA, the Cost-Of-Risk Analysis system The CORA risk model uses data collected about threats, functions and assets, and the vulnerabilities of the functions and assets to the threats to calculate the consequences, that is, the losses due to the occurrences of the threats

17. CORA It is a methodology where the risk parameters are expressed quantitatively and where losses are expressed in quantitative monetary terms CORA uses a two-step process to support risk management. Parameters for threats, functions and assets are validated and refined until the best values are determined

18. CORA CORA then calculates SOL and ALE for each of the threats identified It estimates a single loss value for a threat to an organization, and then multiplies this value by the frequency of the threat occurrence

19. IS Risk Analysis based on a Business Model The IS Risk Analysis Based on a Business Model was developed at the Korea Advanced Institute of Science and Technology This model was developed because traditional risk analysis methodologies had some limitations It takes an asset’s value and then not only bases the analysis on its replacement cost, but also measures the tangible asset’s value from the viewpoint of operational continuity

20. IS Risk Analysis The methodology has four stages During this methodology, the importance level of various business functions of the business model and the necessity level of various IS assets are determined Mathematical formulae are used to calculate ALE for a single threat occurrence on the organization The end result is a quantitative monetary value

21. Criteria for the Framework Whether risk analysis is done on single assets or groups of assets Where in the methodology risk analysis is done The people involved in the risk analysis The main formulae used Whether the results of the methodology are relative or absolute

22. Criteria for the Framework Each criterion has a scaling The scaling indicates the level of a criterion based on certain trade-offs In the end a compliance factor must be selected to indicate how relative the criterion is to a methodology

23. Whether Risk Analysis is done on Single Assets or Group of Assets Criteria can either take on a value of 1 or 2, as follows: 1: if risk analysis is done on individual assets 2: if risk analysis is done on groups of assets In the case of the OCTAVE and CORAS methodology, the risk analysis is done on a single asset If the end result is a single value for a threat scenario that can affect more than one asset, the risk analysis is done on a group of assets, such as in the case of the CORA,ISRAM and IS Business model methodology

24. Where in the Methodology Risk Analysis is Done This criterion explains where in the methodology risk analysis takes place Some preparation, where values for information are estimated, must be done before risk analysis can be performed Some risk analysis methodologies may require more values for different information to be estimated than other methodologies

25. Where in the Methodology Risk Analysis is Done OCTAVE needs values for impact and probability IS Risk Analysis Based on a Business Model needs values for probability, income loss, replacement cost and relative importance of business functions Methodology that needs little preparation and less information is CORA An accurate risk analysis, more preparation means more detailed results, thus ISRAM would be a better option

26. Scale for this Criterion The scale for this criterion is based on a trade-off between time and accuracy. If time is most important 1: Risk analysis done after extensive preparation 2: Risk analysis done after some preparation 3: Risk analysis done after little preparation If accuracy is most important 1: Risk analysis done after little preparation 2: Risk analysis done after some preparation 3: Risk analysis done after extensive preparation

27. The People Involved in the Risk Analysis The people involved in the risk analysis can either be internal or external to the organization CORA uses external risk experts to perform the risk analysis, whereas OCTAVE uses internal personnel exclusively

28. Scale for this Criterion The scale for this criterion is based on a trade-off between cost and expertise If cost is most important, values are as follows: 1: Risk analysis is performed by external experts 2: Risk analysis is performed by external and internal people 3: Risk analysis is performed by internal people If expertise is most important, values are as follows: 1: Risk analysis is performed by internal people 2: Risk analysis is performed by external and internal people 3: Risk analysis is performed by external experts

29. The Main Formulae Used Some methodologies use mathematical formulae while others use an expected value matrix The main formula shows the magnitude of calculations that need to be done, thus indicating the complexity of the risk analysis The scale for this criterion is based on a trade-off between simplicity and accuracy

30. Scale for this Criterion If simplicity is most important, values are as follows: 1: Risk analysis involves extensive mathematical calculations 2: Risk analysis involves some but simple mathematical calculations 3: Risk analysis involves no mathematical calculations If accuracy is most important, values are as follows: 1: Risk analysis involves no mathematical calculations 2: Risk analysis involves some but simple mathematical calculations 3: Risk analysis involves extensive mathematical calculations

31. Whether the Results of the Methodology are Relative or Absolute Some methodologies produce results that are relative. This means that there is no relationship between results and they cannot be compared Other methodologies produce results that are absolute and can be compared The scale for this criterion is based on a trade-off between a methodology providing merely a ranking of risks, or a methodology that can calculate how much greater one risk is over another

32. Scale for this Criterion If merely ranking of risks is most important, values are as follows: 1: Results are comparable 2: Results are not comparable If the differences between risks (how much greater one is over another) are most important, values are as follows: 1: Results are not comparable 2: Results are comparable

33. The Framework for Comparison

34. The Main Formulae Usedin OCTAVE The OCTAVE methodology uses an Expected Value Matrix to determine a risk’s expected value. The main formula is: Loss = Impact/consequence x Probability

35. The Main Formulae Usedin CORAS The CORAS methodology also uses the impact and probability approach. Loss = Impact x Probability

36. The Main Formulae Usedin ISRAM

37. The Main Formulae Usedin CORA ALE = Consequence x Frequency consequence = Σn(individual SOL’s) n the number of single occurrence losses, and SOL = loss potential (worst case monetary value) x vulnerability CORA uses some, but not extensive mathematical calculations. It gets a value of 2 for both simplicity and accuracy

38. The Main Formulae Usedin IS IS Risk Analysis Based on a Business Model uses the following:

39. How the Framework Should be Used The use of the framework is rather simplistic, which allows for use by more organizations Firstly the organization must identify their specific needs. Then, based on the scales of each criterion as defined earlier, the organization must determine values for the specific methodologies they want to compare

40. Strengths Can be applied to various risk analysis methodologies Takes the requirements of an organization into account It uses scales based on different scenarios and trade- offs Can give an indication of which assets and people will be needed for the risk analysis as based on the requirements of the organization

41. Weakness Not taking the customization of a methodology into account The OCTAVE methodology can be tailored to fit the needs of an organization Not all processes have to be performed, which can influence the place where risk analysis fits into the methodology The preparation required can therefore be reduced The risk analysis based on the requirements of an organization

42. Weakness The existence of other criteria, not presented by the framework There are many other risk analysis methodologies, such as CRAMM and there are also baselines, which cover a wider variety of information security aspects, such as the ISO 17799 framework and which can be used to define other criteria

43. Conclusion Numerous methodologies are currently available and many organizations are faced with the daunting task of determining which one to use The goal was to develop an easy-to-use framework that organizations can employ to compare different information security risk analysis methodologies The main benefit lies in the ability to eliminate the majority of methodologies that are unsuitable and to only further investigate the few that remain

44. References A Framework for Comparing Different Information Security Risk Analysis Methodologies ANITA VORSTER And LES LABUSCHANGE BADENHORST, K.P, ELOFF, J.H.P, AND LABUSCHAGNE, L, 1993, A comparative framework for risk analysis methods, Computers & Security CERT COORDINATION CENTER, 2003, The OCTAVE approach, http://www.cert.org/ FREDRIKSEN, R, KRISTIANSEN, M, GRAN, B, AND STOLEN, K, 2001. The CORAS framework for a model-based risk management process, http://coras.sourceforge.net/documents/2002-Safecomp.pdf INTERNATIONAL SECURITY TECHNOLOGY Inc (IST Inc). 2000. Managing risks using CORA, PowerPoint presentation www.ist- usa.com