Signal Detection, Information Theory, and Absolute Judgment Engineering Psychology and Human Performance Park Young Ho Dept. of Nuclear & Quantum Engineering Korea Advanced Institute of Science and Technology February 03 2006 Let me start Good afternoon everyone Today I’d like to present “Signal detection, information theory and absolute judgment of the Engineering psychology and human performance”

Contents 1. Introduction 2. A Model of Human information Processing 3. Signal Detection Theory 4. Information Theory 5. Absolute Judgment 6. Summary and Further Study 7. Reference This slide shows the outline of my presentation. First, introduction , I’d like to talk about a model of human information processing Next Signal detection , information Theory and absolute judgment Finally summary and further study

Introduction A Model of human information processing stages provides a useful framework for analyzing the different psychological processes used in interaction with systems and for carrying out a task analysis. Signal Detection theory (SDT) will deal with the situation in which an observer classifies the world into one of two state: a signal is present or it is absent. The detection process is modeled within the framework of SDT. How the model can assist engineering psychologists in understanding the complexities of the detection process, in diagnosing what goes wrong when detection fails, and in recommending corrective solution. The process of detection may involve more than two states of categorization. At this point , introduce information theory, and then use it to describe the simplest form of multilevel categorization, the absolute judgment task. I’ll talk about a model of human information. This ~ 첫번째줄

A Model of Human information Processing(1) Attention Resources Long-term Memory Working Cognition Sensory Processing STSS Perception Response Selection Execution System Environment (Feedback) Top-Down This picture is a model of human information processing stages. Main stream of model is consist of five stages. first, sensory processing, perception, cognition, response selection and Execution, finally feedback Sensory Processing Information and events in the environment must gain to the brain. All sensory systems have an associated short-term sensory store (STSS) within the brain. Delay time : Visual STSS: 0.5sec, Auditory : 2-4sec Perception Role of the stage is to decode meaning from the raw sensory data. This processing has two important features. 1) Proceeds automatically and rapidly 2) Bottom-up processing: By sensory input Top-down processing : By input from long term memory Cognition and Memory Important distinction between perception and cognition is that cognitive operations require greater time, mental effort, or attention and are carried out by using working memory. Resource limited: conscious activities which transform or retain information Long term memory: It is available to be transferred information in the working memory by learn. Response Selection The understanding of a situation, achieved through perception and augmented by cognitive transformation. Response Execution The latter requiring the coordination of the muscles for controlled motion, to assure that the chosen goal is correctly obtained Feedback Actions are directly sensed by the human or, those actions influence the system, feedback loop will be observable sooner or later. Attention Supply of mental resources Many mental operations are not carried out automatically but require the selective application of these limited resources Bottom-Up Confirmation (intended goal) Continuation (flow of information)

A Model of Human information Processing(2) Sensory Processing Information and events in the environment must gain to the brain. All sensory systems have an associated short-term sensory store (STSS) within the brain. Delay time : Visual STSS: 0.5sec, Auditory : 2-4sec Perception Role of the stage is to decode meaning from the raw sensory data. This processing has two important features. 1) Proceeds automatically and rapidly 2) Bottom-up processing: By sensory input Top-down processing : By input from long term memory Cognition and Memory Important distinction between perception and cognition is that cognitive operations require greater time, mental effort, or attention and are carried out by using working memory. Resource limited: conscious activities which transform or retain information Long term memory: It is available to be transferred information in the working memory by learn. Raw sensory data relayed to the brain must be interpreted, or given meaning, through the stage of perception.

A Model of Human information Processing(3) Response Selection The understanding of a situation, achieved through perception and augmented by cognitive transformation. Response Execution The latter requiring the coordination of the muscles for controlled motion, to assure that the chosen goal is correctly obtained. Feedback Actions are directly sensed by the human or, those actions influence the system, feedback loop will be observable sooner or later. 1)Flow of information can be initiated at any point 2)Flow of information is continuous Attention Supply of mental resources Many mental operations are not carried out automatically but require the selective application of these limited resources.

Signal Detection Theory (Paradigm 1) There are two discrete states of the world: Signal and Noise Signal must be detected by the human operator Two response categories are produced: (1) YES (I detect a signal) (2) NO ( I do not) The combination of two states of the world and two response categories produces the 2 X 2 matrix, generating four classes of joint events Hit, Miss, False Alarm, Correct Rejection This picture is the four outcomes of SDT Signal detection theory is applicable in any situation in which there are two discrete states of the world (signal and noise) that cannot easily be discriminated 두번째읽고 In the process 세번째 읽음 <The four outcomes of SDT>

Signal Detection Theory (Paradigm 2) The SDT model assumes that (Green & Swets, 1996) : (1) Sensory evidence is aggregated concerning the presence or absence of the signal. (2) A decision is made about whether this evidence indicates a signal or not. Neural Evidence (X) : Rate of firing of neurons at a hypothetical “detection center” Critical Threshold: Response Bias (XC) X > XC : Operator Decision (YES) X < XC : Operator Decision (No) P(hit) + P(miss) = 1 P(false alarm) + P(correct reject)=1 This picture is Hypothetical distributions underlying SDT This is noise distribution, this is signal distribution The figure plots the probability of observing a specific value of X given that a noise trial (left cure) or signal trial (right curve) When the absolute probability that X was produced by the signal equals the probability that it was produced by only noise, the signal and noise curves intersect. When the curves intersect, Xc value is critical threshold All X values to the right will cause the operator to respond yes. All to the left generate no responses. Sensitivity of the Picture A is higher than Picture B. Sensitivity is Separation of noise and signal distributions. <Hypothetical distributions underlying SDT> High sensitivity Low sensitivity

Signal Detection Theory Setting the Response Criterion: Optimality in SDT Where is XC “liberal” or “risky” Left ← Hit ↑ False alarm ↑ Β<1 “conservative” Right → Miss ↑ False alarm ↓ Β>1 How to set Beta Ratio of the two curves For a given level of XC This slide shows the setting the response criterion Operator’s conservative or risky behavior is determined by placing the decision criterion Xc. If Xc is placed to the left, operator may be liberal or risky: prone to saying “yes” therefore detecting most of the signals that occurs but making many false alarms. So hit and false alarm are increased If Xc is placed to the right, they may be conservative :saying on most times and making few false alarms but missing many of the signals. Beta is the ratio of neural activity produced by signal and noise at Xc or this is the ratio of the height of the two curves for a given level of Xc. If Xc is placed to the left, beta is less than one. If Xc is placed to the right bets is more than one. So Xc and Bets define the response bias or response criterion. We will first consider the influence of signal probability then payoffs on the optimal setting of beta and finally, human performance in setting beta. Signal Probability Payoffs Human Performance

Signal Detection Theory Signal probability Defines where βopt should be set, but β is set is determined by the observer and must be derived from empirical data. Signal Probability P(S) ↑ βopt↓ (made riskier), Signal Probability P(S) ↓ βopt ↑ (conservative) Payoffs In this case, βopt is maximizing the total expected financial gains or minimizing expected losses. V : Value of desirable events ( hit: H, correct rejection: CR) C : Cost of undesirable events (miss: M, false alarm: FA) When we are setting the response criteria This slide shows the influence of the signal probability and payoffs.

Signal Detection Theory Human Performance in Setting Beta Operator’s need to respond “creatively” by introducing the rare response more often than is optimal, since extreme values of beta dictate long strings of either yes (low beta) or no (high beta). Operator misperceives probabilistic data. There is evidence that people tend to overestimate the probability of rare events and underestimate that of frequent events. When we are setting the response criteria This slide shows the influence of the human performance

Signal Detection Theory The Receiver Operating Characteristic (ROC) Curve This graphic method is used to portray this equivalence of sensitivity across changing levels of bias The ROC curve is useful for obtaining an understanding of the joint effects of sensitivity and response bias on the data from a signal detection analysis. conservative (1) Sensitivity Separation of noise and signal distributions. d’: sensitivity measure (2) Theoretical Representation Of the four value, only two are critical ( hit, false alarm) The ROC curve plots P(h) against P(fa) for different settings of the response criterion The value of beta at any given point along the ROC curve is equal to the slope of a tangent drawn to the curve at that point. Low sensitivity I would like to talk about ROC curve ROC curve is the receiver operation characteristic. Β = slope

Signal Detection Theory Applications of Signal Detection Theory (SDT) Benefits (1)It provides the ability to compare sensitivity and therefore the quality of performance between conditions or between operators that may differ in response bias. (2) it provides a diagnostic tool that recommends different corrective actions. Medical diagnosis Recognition Memory and Eyewitness Testimony Industrial Inspection This slide shows applications of SDT By partitioning performance into bias and sensitivity components, depending on whether change in performance results from a loss of sensitivity or a shift in response bias.

Information Theory How to quantify this flow of information so that different tasks confronting the human operator can be compared Measure task difficulty by determining the rate at which information is presented. Measure processing efficiency, using the amount of information an operator processes per unit of time. Provides metrics to compare human performance across a wide number of different tasks. Information Theory The Number Of Events Probabilities Of events Sequential Constraints and Context I’d like to talk about the information theory In this slide A fundamental issue in engineering psychology is~ Using the information theory We can measure~ We can also ~ Therefore Information theory ~ Also quantification of information is influenced by three variables The number of possible events that could occur event N Probabilities of events The event’s sequential constraints, or the context in which they occur.

Information Theory The number of Events: Probabilities of Events When all alternatives are equally likely, the information conveyed by an event Hs, in bits, can be expressed by the formula. where N is the number of equally likely alternatives. Information theory has a quality of optimal performance. This is based on the minimum number of questions therefore arrives at a solution in a minimum time. Probabilities of Events The probabilistic element of information is quantified by making rare events convey more bits. where Pi is the probability of occurrence of event i. First talk about the number of events Next probability of evet

Information Theory Psychologists are often more interested in measuring the average information conveyed by a series of events with differing probabilities that occur over time. Event A B C D Pi 0.5 0.25 0.125 2 4 8 1 3 0.375 If probabilities of events are different, using this formula. For example there are four events a,b,c,d, with probabilities of 0.5 0.25 0.125 0.125 The computation of the average information conveyed by each event in a series of such events would proceed as follows This value is less than log4=2 if probabilities of events is the same This means ~ Low –probability events convey more information because they occur infrequently. However, the fact that low-probability events are in frequent causes their high-information content to contribute less to the average.

Information Theory Sequential Constraints and Context Redundancy Given a particular context, it may be highly expected, and therefore its occurrence conveys very little information in that context. Absolute probability of the event Pi is now replaced by a contingent probability Pi|X (the probability of event I given context X) Redundancy Defines this potential loss in information where Have is the actual average information conveyed taking onto account all three variable. Hmax is the maximum possible information that would be conveyed by the N alternatives if they were equally likely. The term Redundancy formally Percent redundancy is quantified by the formula

Information transmitted through the system No information transmitted Information Theory Information Transmission of Discrete Signals Concept of information channel How much information is presented to an operator Channel Capacity : How much information is transmitted from stimulus to response Bandwidth : How rapidly information is transmitted. Hs HR HT Hloss Noise Hs HR HT Hloss Noise Information transmitted through the system No information transmitted This slide shows information transmission of discrete signals Concept of information channel is consist of three part First HS: Value of stimulus information HR: Response information HT: Information faithfully transmitted HL: Information loss

Information Theory Stimulus A B C D 2 (0.25) Response Stimulus A B C D 1 2 This slide show the example that quantities Hs,Hr and Hsr are calculated. If each stimulus generates consistently only one response If there is greater dispersion within the matrix, there are more bit within hsr more than hs hs more than ht Response

Absolute Judgment Absolute Judgment: An observer assigns a stimulus into one of multiple categories along a sensory dimension. Inspector of wool quality who must categorize a given specimen into one of several quality levels. Driver who must interpret and recognize the color of a display symbol appearing on his map display. I would like to talk about absolute judgment This is an~ For example Absolute judgment is consist of the these factors I will talk about those factors next slide

Typical human performance in absolute judgment tasks Single Dimensions Channel Capacity (Experimental Results) The level of the late part indicates the channel capacity of the operator. (2~3bits) Errors began to occur (HT<HS) George miller Edge Effect (Experimental Results) Stimuli located in the middle of the range of presented stimuli are generally identified with poorer accuracy than those at extremes (Shiffrin & Nosofsky, 1994) 1 2 3 4 5 Hs Ht Perfect Performance Human Performance Typical human performance in absolute judgment tasks

Number of combined dimensions (Increasing Hs) Absolute Judgment Multidimensional Judgment Most of our recognition is based on the identification of some combination of two or more stimulus dimensions rather than levels along a single dimension. Orthogonal dimensions The level of the stimulus on one dimension can take on any value, independent of the other For example, Weight and Hair color As more dimensions are added, more total information is transmitted, but less information is transmitted per dimension. 1 2 3 4 5 Number of combined dimensions (Increasing Hs) Total Ht Perfect Performance Human Performance 6 7 Bits/dimension Human performance in absolute judgment of multidimensional auditory stimulus

Absolute Judgment Correlated Dimensions The level on one constrains the level on another. For example, Height and weight As more dimensions are added, the security of the channel improves, but HS limits the amount of information that can be transmitted. Orthogonal dimensions maximize HT, the efficiency of the channel. Correlated dimensions minimize Hloss; that is, they maximize the security of the channel. Number of combined dimensions (Increasing Hs) Information transmitted, Ht Hs Hloss Human performance in absolute judgment of multidimensional auditory stimulus

Summary and Further Study The model of that we have described for Human information processing is not a computational one that can provide estimates, say, of the time required to perform certain tasks, or the error rates expected. Rather, it provides a general framework for analyzing human performance. We have seen how people classify stimuli into two levels along one dimension, several levels along one dimension, and several levels along several dimensions. Signal detection was characterized by the probabilistic element of decisions under uncertainty. Further Study Engineering Psychology and Human Performance Manual Control Attention, Time-Sharing, and Workload Stress and Human Error Complex Systems, Process Control, and Automation

Reference Engineering Psychology and Human Performance Third Edition, Wickens & Hollands