1. Risk Assessment: A Practical Guide to Assessing Operational Risk
Chapter 4:
Defining Risk Assessment Criteria

2. Risk Assessments: Defining Risk Assessment Criteria
Objectives
Introduce Risk Assessment Criteria
Discuss Risk Scoring Systems and their Components
Describe How to Select, Develop and Apply Risk Criteria
Discuss Risk Reduction and the Hierarchy of Controls in Risk Estimation
Review Acceptable Risk Level Criteria

3. Introduce Risk Assessment Criteria
Before operational risks can be assessed, an organization must first establish and define the risk criteria from which to measure and score.
Such criteria must be clearly defined and communicated by an organization to consistently evaluate operational risks, and make proper risk-based decisions.
A number of existing risk criteria measures are available, however, it is essential that an organization carefully select and/or develop its own risk criteria to reflect its values, goals, industry setting, and overall culture.

4. Defining Risk Criteria
Risk criteria are the reference points against which the significance of risk are evaluated and measured. Such criteria are derived from the organization’s culture and industry, external and internal context, applicable laws, standards and other requirements. In general, risk criteria should include a risk scoring system that includes risk factors, defined scales of risk levels and a risk matrix for an organization to measure risk for the purpose of prioritizing and making proper decisions.

5. Defining Risk Criteria
Risk is defined as a hazard’s estimated likelihood (or probability) of occurrence, and the resulting severity of consequences. Thus, risk criteria must begin with these two risk factors: likelihood and severity. Several other risk factors can be added to further define risk such as exposure or frequency of exposure, duration of exposure, vulnerability, failure detectability, control reliability, and prevention effectiveness.

6. Risk Scoring Systems
The primary purpose of risk assessment is to identify hazards, and assess and reduce their risk to an acceptable level. To achieve this, a measurement system that includes a baseline (an organization’s acceptable risk level) and a method of scoring (a risk scoring system) must be established.

7. Risk Scoring Systems
Manuele was the first to use the term ‘risk scoring system’ for operational risk assessments in his 2001 book. He states that two-dimensional risk assessment matrices using likelihood (L) of event occurrence and severity of consequence (S) have been commonly used in risk assessment exercises. However, risk scoring systems with three or four risk factors are becoming more common, adding a third or fourth factor such as failure detectability, control effectiveness, vulnerability or other.

8. Risk Scoring System Components
Risk Factors - Specific risk factors used to measure risk such as severity, likelihood or probability, frequency, duration, failure detectability, control effectiveness, vulnerability, or other risk measures.
Risk Levels - Specified risk levels or categories for each risk factor (typically 3 to 6 levels)
Risk Values - Defined qualitative, quantitative or semi-quantitative values for risk levels
Risk Actions - Decision guidelines or action required for each risk level
Risk Screening and Communication Tools - A risk assessment matrix, graph or risk priority numbers (RPNs) used to measure, screen, compare, prioritize as well as communicate risk within an organization.

9. Risk Scoring System Components

10. Risk Assessment Matrices
A key part of a risk scoring system is the risk assessment matrix. A matrix helps visualize and communicate risk levels to decision makers by providing a means for categorizing combinations of likelihood and severity and their risk levels. They are often used as a screening tool when there are many risks to evaluate.

11. Risk Assessment Matrices

12. Defining Risk Values
Risk assessment models and their matrices may be classified as qualitative, semi-quantitative or quantitative. When defining the risk criteria and risk scoring system to be used, stakeholders must take into consideration the level of detail desired, and data and resources available.

13. Defining Risk Values
Qualitative risk models are based on qualitative or subjective descriptions rather than numerical or statistical data, and require less precise information to be developed and used. Qualitative risk models define severity of consequence, likelihood and level of risk using descriptive words such as “high”, “medium” and “low” which are evaluated according to qualitative criteria.

14. Defining Risk Values
Qualitative risk models

15. Defining Risk Values
Semi-quantitative risk models use qualitative data; however the values are expressed as numerical risk ratings using a formula to produce a risk level or score. Risk level scores produced can be linear or logarithmic based on the formula selected. One advantage of a semi-quantitative model is that more precision can be given by adding definitions that include some numerical ranges for severity of consequences and likelihood of occurrence.

16. Defining Risk Values
Semi-quantitative risk models

17. Defining Risk Values

18. Defining Risk Values
Quantitative risk models use data to define values for severity of consequences and likelihood of occurrence, and produce risk level values in specific numerical units.
As described in ANSI Z690.3, ‘full quantitative analysis’ may not be possible or desired if there is insufficient information or data available about the system or activity to be analyzed, or the efforts required exceed the needs of the assessment.

19. Risk Factors
Risk factors are the components of risk derived from an identified hazard that are estimated and measured to produce a risk score. Risk assessments generally have two-dimensional risk scoring systems, which use two risk factors such as severity of consequence (S), and likelihood (L) or probability (P) of occurrence.

20. Risk Factors
Risk factors used in various three and four dimensional models.
Exposure (E) is used as a general measure of exposure events/units.
Frequency of Exposure (F) is used as a number of exposure events for a unit of time.
Time Duration of Exposure (T) is used as a time period that a single exposure occurs.
Vulnerability (V) is sometimes used in security threat analyses, and generally refers to weaknesses in a system that are factored into the risk estimation.
Detection of Failure (D) is used in many FMEA models as a third risk factor in the risk level scoring system. The detection rating is based on an estimate of how easily the potential failure could be detected prior to its occurrence.
Control Reliability (CR) is used in machine risk assessments and factors the reliability of a selected control into the risk estimation.
Prevention Effectiveness (PE) is a risk factor sometimes used in FMEA and other methods to evaluate a controls effectiveness in preventing a failure from occurring.

21. Risk Levels
Risk levels or categories are the defined graduated levels of increasing risk established for each risk factor; and for risk scores produced from combined risk factors used in the risk assessment matrix.

22. Risk Scoring
Risk scores are produced by combining risk factors. When three or more risk factors are used, a risk priority number (RPN) is produced. Risk priority number (RPN) - a semi-quantitative measure of criticality obtained by multiplying numbers from rating scales (usually between 1 and 10) for consequence of failure, likelihood of failure and ability to detect the problem. (A failure is given a higher priority if it is difficult to detect.)
Risk Priority Number = Severity x Likelihood x Detection

23. Risk Scoring
The use of three and four risk factors systems should be carefully examined. As indicated by Manuele, a four factor risk scoring system can distort or dilute severity level of a particular risk if all four factors are given equal weight. To more accurately score risk levels, Manuele proposes that severity receive a 50% weighting to reflect the impact severity has on incident outcomes. In the following equation, the rating for occurrence probability and rating for frequency of exposure are added together and then multiplied with severity. Severity x (Probability + Frequency of Exposure) = Risk

24. Severity of Consequence
Consequences are the results, outcomes or losses of an event caused by a hazard(s). Consequences most often refer to the damage or harm caused to people, assets/property or the environment. As a primary risk factor, the ‘severity levels’ of consequences to be used in an assessment must be determined upfront, during the development of the context. This will include the types of consequences and levels of severity. Risk should be evaluated for the worst credible case rather than worst conceivable risk

25. Severity of Consequence
The completed severity categories and their descriptions are used in the risk matrix.
It is important that severity categories are clearly defined so that consequences can be consistently ranked or scored by the risk assessment team.

26. Likelihood of Occurrence
Likelihood is sometimes referred to as probability in risk management terminology. Although these terms are often used interchangeable, there are distinctions to take into consideration. Likelihood is the chance of an event or something happening, generally expressed qualitatively. Probability is a quantitative or numerical measure of the chance of something happening expressed as a percentage. Both can be used successfully. Definitions for likelihood or probability must be selected or developed and provide a clear understanding of their meaning for each level.

27. Likelihood of Occurrence
Three methods are used to estimate likelihood or probability for risk assessments. These include the use of: 1) historical data; 2) predictive techniques; and 3) expert opinion.

28. Exposure
Exposure is an indication of the extent to which the organization is subject to the consequences based of the amount of exposure in numbers. Some risk assessments include exposure as a third risk factor to severity and likelihood. Exposure can be measured as the frequency of an event or exposure, its duration, and/or the assets exposed to risk.

29. Exposure Some of the variables for exposure might include:
the number of employees or people exposed
how frequent an activity is performed
the miles driven or number of vehicles used in transportation
the number of customers or products for a product risk assessment
the number of locations or facilities for a property risk assessment

30. Risk Reduction and the Hierarchy of Controls
The Hierarchy of Controls is a model for identifying the risk reduction effectiveness of control types. It is defined by ANSI Z590.3, Prevention through Design as:
A systematic approach to avoiding, eliminating, controlling, and reducing risks, considering steps in a ranked and sequential order, beginning with avoidance, elimination, and substitution. Residual risks are controlled using engineering controls, warning systems, administrative controls, and personal protective equipment.

31. Risk Reduction and the Hierarchy of Controls
The most effective risk reduction is achieved through avoidance of the risk or elimination by design or redesign. Lower level controls should only be selected after practical applications of higher level controls are considered.

32. Risk Reduction and the Hierarchy of Controls
Two Stage Iterative Approach to the Hierarchy of Controls and Risk Reduction from ANSI B – Safety of Machinery

33. Protection Factor
Protection factors provided by both existing controls and those being proposed can be selected and used in a risk assessment. When incorporating a control or protection factor, the risk estimate is adjusted to reflect the level of protection provided. An example protection factors formula and scale are provided.

34. Acceptable and Unacceptable Risk Levels
Acceptable risk level can be defined as the risk level an organization is willing to tolerate in its current context. Acceptable risk levels, as well as unacceptable levels tend to be lowered as an organization becomes more effective in their risk management efforts, reducing risk and improving control technologies.

35. Documenting Risk
Risk assessment results should be well documented to demonstrate the methods, communicate the results and to be referred to and understood by different people at different times. In ANSI Z , 7.12, Document the Results, it suggests documenting the names, titles, and qualifications of the risk assessment team, the methods, hazards identified, risks, controls and follow-up actions.

36. Documenting Risk
A risk register is one well accepted method of documenting the risk assessment and its results.

37. Communicating Risk Criteria
Defining risk criteria has little value if it is not effectively communicated to all affected stakeholders. Communication must start from the beginning, during the context phase throughout the process and include monitoring and verifying risk reduction results.

38. Conclusion
At its core, risk assessment is governed by the specific risk criteria established by its stakeholders. The importance of well-defined risk criteria within a risk scoring system cannot be over emphasized. However, SH&E professionals should always keep in mind the ultimate purpose of risk assessment which is to ‘reduce risk to an acceptable level’.