Origins of Signal Detection Theory Problem in Psychophysics Thresholds: is sensitivity discrete or continuous? Sensitivity confounded with response bias

Thresholds Solution: detection theory (engineering)

Signal Detection Theory Response Yes No Hit (H) Miss (M) False Alarm (FA) Correct Rejection (CR) Signal State of the World Assumes two possible states of the world: signal and noise (see Figure 1). This assumes a crisp, traditional set: events either are or are not signals. Noise

Assumptions of Signal Detection Theory Noise is always present (i.e. in the nervous system and/or in the signal generating system) The noise is normally distributed with σ2 = 1 For Gaussian model When a signal is added to the noise, the distribution is shifted upward along the sensory dimension. Variance remains constant (equal variance model).

Assumptions of Signal Detection Theory Observers are both sensors and decision makers To evaluate the occurrence of an event, observers adopt a decision criterion Sensitivity and Response Bias are independent Statistical Theoretical Empirical

Distribution of Noise and Signal + Noise

Sensitivity d’ = zH - zF d’ Task Person >3.5 very easy very sensitive 2.6-3.5 moderately easy moderately sensitive 1.6-2.5 moderately difficult moderately insensitive <1.5 very difficult very insensitive

Relation of d’ to Other Statistics If μn=0 and σn=1 (i.e., if the N distribution is unit normal) then the ROC function, in its most general form, is

Testing a Mean From One Distribution

Relation of d’ to t-test

Comparing Means from Two Distributions

Standardized Mean Difference Effect Size

Response Bias  = f(SN)/f(N) c = -.5(zH + zF) Lenient: 0-1 Unbiased: 1 Conservative: 1- 8 c = -.5(zH + zF) Lenient: <0 Unbiased: 0 Conservative: >0

Three values of  1 2 3 Sensory magnitude (X) P(event|x) N SN

Receiver Operating Characteristic (ROC)

ROC Curves: Sensitivity

ROC Curves: Response Bias

ROC Curve in Z-score form 1 ZH 1 ZFA

ROC for σ2N = σ2SN 3 ZH -3 -3 3 ZFA

What is Independence? Statistical: P(A|B)=P(A) PB|A)=P(B) Theoretical/Logical: β can vary independently of d’ Empirical: experimental evidence is consistent with the independence assumption (e.g. Form of empirical ROC)

Three values of  1 2 3 Sensory magnitude (X) P(event|x) N SN

ROC Curve in Z-score form 1 ZH 1 ZFA

ROC for σ2N = σ2SN 3 ZH -3 -3 3 ZFA

What if both the mean and variance Change?

Alternative Sensitivity Measures Az: Area under the ROC (e.g., see Swets,1995, ch. 2-3; Swets & Pickett, 1982) Range: from .5—1.0 Underlying distributions can have unequal variances Assumes that the underlying distributions can be monotonically transformed to normality ZH= a + bZF

Area under the ROC

‘Non-parametric’ Measures: Sensitivity Not really non-parametric: No distribution assumed, but follows a logistic distribution (Macmillan & Creelman, 1990)

‘Non-parametric’ Measures: Response Bias For applications to vigilance, see See, Warm, Dember, & Howe (1997)

What if the Situation is More Complex? Response 1 2 3 4 5 6 7 State of the World

Identification and Categorization 1  2  3  4  5  6 Response 1 2 3 4 5 6 7 x

Fuzzy Logic Traditional Set Theory: A ∩ A = 0 Fuzzy Set Theory: A ∩ Ā ≠ 0 One assigns non-binary membership, or degrees of membership, to classes of events (fuzzification). Traditional Set Theory: Set membership is mutually exclusive and exhaustive: elements either are or are not members of a set. Fuzzy Set Theory: Events can simultaneously be and not be members of a set.

Elements of Fuzzy Signal Detection Theory Events can belong to the set “signal” (s) to a degree ranging from 0 to 1 Events can belong to the set “response” (r) to a degree ranging

Computation of FSDT Measures Select mapping functions for signal & response dimensions Assignment of degrees of membership to the four outcomes (H, M, FA, CR) using mixed implication functions. Compute fuzzy Hit, Miss, False Alarm, and Correct Rejection Rates Compute detection theory measures of sensitivity and response bias

1. Mapping Functions To assign degrees of (s, r) membership to events, all possible states of the world and each possible response must be evaluated using a mapping function. For the set s a mapping function relates the signal value (s) to a variable (or set of variables) that describe the state of the world (see Figure 3). A mapping function for the set r relates the response value (r) to a response variable (or set of variables). For instance, a mapping function could be based upon confidence ratings of signal presence, a technique used in traditional SDT (Green & Swets, 1966;MacMillan & Creelman, 1991). Note that the mapping function can be discrete or continuous, and can be derived empirically, theoretically, or based upon legal or industry standards. Either s or r (or both) can be fuzzy. The measurement challenge is to map s to the actual states-of-the-world. The derivation of the function may be theoretically based, but the reliability and validity of these functions should be established empirically.

2. Assignment of Set Membership to Categories Mixed Implication Functions H = min (s,r) M = max (s-r, 0) FA = max (r-s, 0) CR = min (1-s, 1-r) When r>s, some degree of FA membership will result, since the response is stronger than the actual degree of signal. Similarly, when r<s, some degree of miss membership results, since the response is less than the degree of “signalness” of the event. ●Performance is best when r=s, as the observer’s responses are well mapped to the degree of signal present. This constitutes an ideal observer. ●Note that Parasuraman et al. (2000) did not claim these functions to be the only possible implication functions. The form of the mapping function can change with the domain of interest.

3. Computation of Fuzzy Hit and False Alarm Rate H= Σ(Hi)/ Σ(si) for i=1 to N M = Σ(Mi)/ Σ(si) for i =1 to N FA = Σ(FAi)/ Σ(1-si) for i=1to N CR = Σ(CRi)/ Σ(1-si) for i= 1 to N Once fuzzy hit and false alarm rates are computed, the standard formulas for SDT indices are applied

Truth Table for FSDT Data Hit FA Miss CR .83 1.00 .17 .50 .33 .67 .16

‘Perfect’ Performance s r Hit FA Miss C’R .83 1.00 .17 .50 .33 .67 .16

‘Hitness’ and ‘False Alarmness’ Miss CR .83 1.00 .17 .50 .33 .67 .16

‘Hit and Miss’ s r Hit FA Miss CR .83 1.00 .17 .50 .33 .67 .16

Fuzzy Stimulus and Response: Duration Discrimination 1 2 3 4 5 6 7 20 msec 200 220 240 260 280 300 320 80 msec 360 440 520 600 680

Comparison of Fuzzy and Crisp ROC Curves

Comparison of Fuzzy and Crisp ROC Curves

Response Time as a Function of Degree of Stimulus Criticality 1100 1000 900 Response Time (msec) 800 Transition hh 700 hl 600 lh 500 ll 1 2 3 4 5 6 7 1 Stimuli

Reaction Time as a Function of Stimulus Value: 80 msec Discrimination

Reaction Time as a Function of Stimulus Value: 20 msec Discrimination