1. An Model for Situation Assessment -from MC Kim’s paper- 2006. 09. 18 Hyun-Chul Lee

2. References MC Kim & PH Seong, An analytic model for situation assessment of nuclear power plant operators based on Bayesian inference, RESS Vol.91(2006), pp. 270-282. MC Kim, Development of a Quantitative Safety Assessment Method for Nuclear I&C Systems Including Human Operators, Doctorial Thesis, KAIST, 2004.

3. Situation Assessment vs. Situation Awareness Situation Assessment –To try to understand what is going on in the plant Situation Awareness –The measure for how correctly operators understand the situation Endsley pointed out, situation awareness is a state of knowledge while situation assessment is the process of achieving, acquiring, or maintaining situation awareness.

4. Situation Awareness/Assessment Models Qualitative Model (Descriptive Model) –Describes how people process or interact with the environment to attain their situation awareness –Useful to understand the process of SA during retrospective event analyses Endsley : Information Processing Approach Bendy and Meister : Activity Theory Approach Adams et. Al. : Ecological Approach Quantitative Model (Prescriptive Model) –Useful to predict what will happen in various situations Miao et. Al. : Computational SA model for NPP operators, IEEE SMC (1997) MC Kim : Based on Bayesian inference (2006)

5. Proposed Model – Situation Assessment Process : Description 1.Abnormal or accident situation occurs. 2.Operators recognize it by onset of alarms. 3.Operators read the relevant indicators. 4.Operators try to establish their situation models. At this point, operators usually also consider the possibility of sensor or indicator failures. 5.If operators receive other alarms, operators will read the relevant indicators. Even if operators do not receive other alarms, operators will probably decide to monitor other indicators to confirm their situation models. 6.Regardless of why they monitor other indicators, the observations they make will alter their situation models accordingly.

6. Proposed Model – Situation Assessment Process : Simplification Plant has finite situations so that NPP operators consider only the finite representative states of the plant. The mental model of an NPP operator can be modeled using the rules on the dynamics of the plant for the representative states of the plant.

7. Proposed Model – Modeling of the Mental Model : Assumptions 1.Each representative state of an NPP corresponds to an accident or a transient, and it is assumed that different kinds of representative states of the plant can be modeled to be mutually exclusive. (We have mutually exclusive representative states of the plant.) 2.It is assumed that operators have deterministic rules. Because the behavior of the indicators under clearly specified accident scenarios can be determined clearly, there is a strong expectation that operators' rules are also deterministic.

8. Proposed Model – Modeling of the Mental Model X indicates the representative states of the plant, X={x 1,x 2,…,x l } Yi's indicate various indicators Zi's indicate various sensors

9. Proposed Model – Modeling of the Mental Model If the state of the plant is x k then the value or the trend of the indicator Y i is expected to be y ij. This kind of rule can be collected from interviews with NPP operators or through simulator simulations. The deterministic rules can be described mathematically using conditional probabilities, as follows :

10. Proposed Model – Situation Assessment based on Bayesian inference We assume that NPP operators use Bayesian inference to process inco ming information. If the operators observe y ij on the indicator Y i, the probability of a state of the plant x k can be revised as follows:

11. An Example the initial probability distribution for the four representative states of the plant recognized by the NPP operator is assumed to be as follows: the NPP operator is assumed to believe that all seven sensors have an equal unavailability, 0.001, and that each sensor has three failure modes, fail-high, stuck-at-steady-state, and fail-low. Therefore, Zi's are given as follows:

12. An Example the NPP operator believes that the probability distribution for Zi's i.e. p(Zi)'s, is given as follows: The NPP operator has rules on the dynamics of plant, which represent his mental model, as shown in Table 1.

13. An Example Based on Table 1, conditional probability tables for the seven indicators in the Bayesian network can be determined. As an example, the conditional probability table for the node Rx_Power is given in Table 2.

14. An Example - Concerns 1.the number of plant states and information sources. It does not seem reasonable to assume that NPP operators actually consider all of the infinite plant states and all of the information sources in real NPPs. Therefore, we think that certain key information sources that are necessary for the diagnosis of a finite number of important plant states can be identified through interviews with NPP operators. 2.the size of the Bayesian networks. In some Bayesian network, the number of data necessary for the conditional probability tables increases exponentially as the number of nodes and their states increase. For a Bayesian network to represent the mental model of an NPP operator, however, the number of data increases proportionally, not exponentially, because each information source has a one-to-one relation with the operator's recognition of the situation.

15. An Example - Description of the Situation LOCA with the CCF of pressurizer pressure sensors in a WH 900MWe PWR the Compact Nuclear Simulator (CNS) in KAERI was used. From the simulation, it was found that the plant protection system (PPS) will not generate an automatic reactor trip signal and that the engineered safety feature actuation system (ESFAS) will not generate an automatic safety injection actuation signal due to the CCF of pressurizer pressure sensors. In this situation, operators have to correctly understand the state of the plant as well as manually actuate rector trip and safety injection.

16. An Example – Probable Situation Assessment At the time of 49s, the operators receive the containment radiation high alarm. an operator will move to the containment radiation indicator, observe that the containment radiation is increasing. the operators might consider two possibilities for the alarm: either a failure of containment radiation sensors or a LOCA. If the operators receive the information that the pressurizer pressure does not change due to the CCF of pressurizer pressure sensors after the observation that the containment radiation is increasing, the operators might then believe that there was a failure of containment radiation sensors in normal operation. However, if, after observing that the containment radiation is increasing and the pressurizer pressure does not change, the operators also receive the information that the reactor power is decreasing, then the operators might believe that this odd behavior indicates that there is a real problem in the plant. In that event, the operators might consider the possibility that a LOCA has occurred.

17. An Example –Situation Assessment of the proposed model The initial probability distribution for the four representative states of the plant recognized by the operator is given as follows: p(X)={0.9997, 0.0001, 0.0001, 0.0001} After observing that containment radiation is increasing, the situation model of the operator changes as shown in Fig. 5. The probability distribution for the four representative states of the plant recognized by the operator p(X) is given as follows: p(X)={0.5001, 0.4998, 0.00005, 0.00005}. This means that the operator considers two states of the plant, normal operation and the occurrence of a LOCA, as having an almost equal probability of occurrence, and does not consider other states as being as likely to occur.

18. An Example –Situation Assessment of the proposed model After observing that the pressurizer pressure does not change due to the CCF of pressurizer pressure sensors, the situation model of the operator changes. The p(X) and the probability distribution for the states of the containment radiation sensor p(Z 6 ) after the observation are given as follows: Eq. (13) means that the operator considers only the state of normal operation as being likely. Eq. (14) means that the operator thinks that the containment radiation sensor is in the fail-high failure state.

19. An Example –Situation Assessment of the proposed model After observing that the reactor power is decreasing, the situation model of the operator changes. The p(X) and p(Z 6 ), the probability distribution for the states of the reactor power sensor p(Z 1 ), and the probability distribution for the states of the pressurizer pressure sensor p(Z 3 ) after the observation are given as follows: Eq. (15) means that the operator suddenly considers the occurrence of a LOCA as being most likely. Eqs. (16), (17) and (18) mean that the operator thinks that the containment radiation sensor and the reactor power sensor are in the state of normal operation but that the pressurizer pressure sensor is experiencing stuck-at-steady-state failure.

20. Discussion 1.Comparison with the existing model –Maio MaioKim Operator’s RuleProbabilisticDeterministic Characteristic of SituationsIndependentMutually exclusive Decision on Sensor failureRepetitive alarmNew information Even though the two methods try to explain the situation assessment of NPP operators using the Bayesian inference, the basic assumptions and structures are completely different.

21. Discussion 2.Addressable Features of the Situation Assessment of NPP Operators –Bayesian inference provides a mathematical basis for diagnostic processes by considering the relations between the causes and effects and their frequencies. In this sense, the proposed model provides a description for the situation assessment of NPP operators when they assess the situation most logically. –Even though some incorrect information from the failed sensors is provided to NPP operators, they are usually able to correctly recognize the abnormal situation and identify the failed sensors. the proposed model provides a quantitative description for the recovery of information by NPP operators. –the proposed model can provide quantitative descriptions for similarity matching and frequency gambling.

22. Discussion Similarity Matching (SM) Table 4 shows a trend indicating that, the more the perceived situation of the operator matches his rules on the dynamics of the plant for a representative state of the plant, the higher the probability for the representative state of the plant. Generally, the proposed model shows a similar trend, and this trend can be used as a quantitative description for similarity matching.

23. Discussion Frequency Gambling (FG) The linear relation of the proposed model between the frequency of a representative state of the plant and the probability for the representative state of the plant can be used to provide a quantitative description for frequency gambling. the same occurrence frequencies for a LOCA and an SGTR accident occurrence frequency of an SGTR accident 3 times higher than that of a LOCA or an SLB accident Initial Probability Distribution p(X)={0.9997, 0.0001, 0.0001} p(X)={0.9997, 0.00006, 0.00018, 0.00006}. After the observation of the decrease in Rx power p(X)={0.333456, 0.333256, 3.34×10 −5 } p(X)={0.294237, 0.176436, 0.529309, 1.77×10 −5 }. Where, p(X)={Normal, LOCA, SGTR, SLB}

24. Discussion 3.Towards a quantitative model for human operators 1)Optimistic Model - The proposed model provides the most logical results of situation assessment, and therefore can be considered to be too optimistic. a human operator would tend to be more conservative and sometimes behave quite illogically. -> Modification of the Bayesian probability equation 2)Uncertainty of data - To apply the proposed model, we need to collect data from NPP operators. A lot of uncertainties will be unavoidably associated with the probability distributions for the states of sensors and the recognized probability distributions for the representative states of the plant. The data problem is also an important issue for the proposed model, as for other models in HRA.

25. Discussion 3.Towards a quantitative model for human operators 3)Finite states and rules - Finding rules on the dynamics of the plant for other abnormal situations is not feasible, because no NPP operator can establish rules for a state that is not clearly defined. It seems that the best we can do is to clearly define the boundaries of the model, and include only a finite number of clearly defined representative states of the plant. 4)No Biases - Human operators have various biases such as the overconfidence bias, anchoring heuristic, and confirmation bias. However, the proposed model, which is an analytic model, does not consider such biases. Therefore, quantitative methods that take such biases into account have to be developed.

26. Discussion 3.Towards a quantitative model for human operators 5)Continuous Probability Update - Usually, human operators continuously receive information without updating the probability distribution for the states of the plant, and then suddenly update the probability distribution for the states of the plant when they feel that such an update is necessary. We do not know how to determine the moment when human operators feel an update is necessary. 6)Capacity of short term memory - The short-term working memory of human operators has a finite capacity. Therefore, the number of observations that human operators can memorize should be determined and reflected in the proposed model.

27. Conclusion As the first step in developing a quantitative model for the situation assessment of NPP operators, an analytic model is developed In the proposed model, NPP operators' rules on the dynamics of the plant for the representative states of the plant are used to represent the mental model of NPP operators, and the probability distribution for representative states of the plant and the probability distributions for the states of sensors are used to represent the situation model of NPP operators. Bayesian inference is used to describe how the incoming information affects the situation model of NPP operators. The proposed model provides quantitative descriptions for several important features of the situation assessment of human operators, including the from-effects-to-possible-causes relation, the recovery of information (identification of failed sensors), and the similarity matching and frequency gambling. This work has also identified limitations of the proposed model and has noted.