10 / 31 Outline Perception workshop groups Signal detection theory Scheduling meetings

Detection experiment Question –How sensitive is an observer to a sensory stimulus; for example, light?

Detection experiment Question –How sensitive is an observer to (for example) light? Classic experiment –Yes/No task

Detection experiment Question –How sensitive is an observer to (for example) light? Classic experiment –Yes/No task –Measure threshold intensity needed to have 50% hits

Threshold

Jane Nancy

Summary of results Thresholds –Jane = 20 –Nancy = 25

Summary of results Thresholds –Jane = 20 –Nancy = 25 False alarm rates –Jane = 51% –Nancy = 18.7%

Look at one intensity level I = 25

Jane’s Hit Rate P(H) =.84

Nancy’s Hit Rate P(H) =.5

Look at one intensity level I = 25 –Jane Hit rate: P(H) =.84

Look at one intensity level I = 25 –Jane Hit rate: P(H) =.84 False alarm rate: P(FA) =.51

Look at one intensity level I = 25 –Jane Hit rate: P(H) =.84 False alarm rate: P(FA) =.51 –Nancy Hit rate: P(H) =.5

Look at one intensity level I = 25 –Jane Hit rate: P(H) =.84 False alarm rate: P(FA) =.51 –Nancy Hit rate: P(H) =.5 False alarm rate: P(FA) =.187

Signal detection theory terms Hits - p(H) –Proportion of “yes” responses when signal is present

Signal detection theory terms Hits - p(H) –Proportion of “yes” responses when signal is present Misses - p(M) –Proportion of “no” responses when signal is present

Signal detection theory terms Hits - p(H) –Proportion of “yes” responses when signal is present Misses - p(M) –Proportion of “no” responses when signal is present False alarms - p(FA) –Proportion of “yes” responses when signal is not present

Signal detection theory terms Hits - p(H) –Proportion of “yes” responses when signal is present Misses - p(M) –Proportion of “no” responses when signal is present False alarms - p(FA) –Proportion of “yes” responses when signal is not present Correct rejections - p(CR) –Proportion of “no” responses when signal is not present

Relationships between terms P(H) + P(M) = 1

Relationships between terms P(H) + P(M) = 1 P(FA) + P(CR) = 1

Relationships between terms P(H) + P(M) = 1 P(FA) + P(CR) = 1 Only need to specify P(H) and P(FA)

Extreme detection strategies Most liberal (always say yes)

Extreme detection strategies Most liberal (always say yes) –P(H) = 1, P(FA) = 1

Extreme detection strategies Most liberal (always say yes) –P(H) = 1, P(FA) = 1 Most conservative (always say no)

Extreme detection strategies Most liberal (always say yes) –P(H) = 1, P(FA) = 1 Most conservative (always say no) –P(H) = 0, P(FA) = 0

Signal Detection Theory

Assume an internal measure of signal strength.

Signal Detection Theory Assume an internal measure of signal strength (X). –E.g. firing rate of ganglion cell

Signal Detection Theory Assume an internal measure of signal strength (X). –E.g. firing rate of ganglion cell X is corrupted by noise

Signal Detection Theory Assume an internal measure of signal strength (X). –E.g. firing rate of ganglion cell X is corrupted by noise –E.g. random variations in firing rate

Signal Detection Theory Assume an internal measure of signal strength (X). –E.g. firing rate of ganglion cell X is corrupted by noise –E.g. random variations in firing rate When signal is not present, X = X 0 + N

Signal Detection Theory Assume an internal measure of signal strength (X). –E.g. firing rate of ganglion cell X is corrupted by noise –E.g. random variations in firing rate When signal is not present, X = X 0 + N When signal is present, X = X S + N

o Firing rate when signal is present o Firing rate when signal is not present

Criterion Set a criterion level, C

Criterion Set a criterion level, C If X > C –Report a signal

Criterion Set a criterion level, C If X > C –Report a signal If X < C –Report no signal

o Firing rate when signal is present o Firing rate when signal is not present C=20, Liberal criterion

Liberal criterion = High hit rate

Liberal criterion = High false alarm rate

o Firing rate when signal is present o Firing rate when signal is not present C=30, Conservative criterion

Conservative criterion = Low hit rate

Conservative criterion = Low false alarm rate

Probability distribution on X (no signal)

Probability distribution on X (signal)

Liberal criterion

Conservative criterion

ROC curve

A B C No signal Signal

A B C

A B C

A B C No signal Signal

A B C

A B C

Determinants of performance No signal Signal

Determinants of performance XNXN XSXS No signal Signal

Determinants of performance XNXN XSXS ∆X No signal Signal

Determinants of performance XNXN XSXS 1. Difference in average strength of Signal measure ∆X = X S - X N ∆X No signal Signal

Determinants of performance 1. Difference in average strength of Signal measure ∆X = X S - X N 2. Amount of noise  ∆X  No signal Signal

Determinants of performance 1. Difference in average strength of Signal measure ∆X = X S - X N 2. Amount of noise  3. Sensitivity d’ = ∆X /  ∆X  No signal Signal

D’ determines which ROC curve your data will fall on

d’ =.83 d’ = 1.2 d’ = 2.5 D’ determines which ROC curve your data will fall on

Criterion determines where your data will sit on an ROC curve

Conservative criterion Liberal criterion Criterion determines where your data will sit on an ROC curve

Measuring sensitivity

Pick a stimulus level for a yes / no task

Measuring sensitivity Pick a stimulus level for a yes / no task Measure hit rate and false alarm rate

Measuring sensitivity Pick a stimulus level for a yes / no task Measure hit rate and false alarm rate Use p(H) and p(FA) to calculate d’

Measuring sensitivity Pick a stimulus level for a yes / no task Measure hit rate and false alarm rate Use p(H) and p(FA) to calculate d’ d’ = absolute measure of sensitivity

Blood test example Get a blood test for level of protein A.

Blood test example Get a blood test for level of protein A. Doctor says that test is positive for liver cancer.

Blood test example Get a blood test for level of protein A. Doctor says that test is positive for liver cancer. Doctor recommends surgery to collect tissue sample for biopsy.

Blood test example Get a blood test for level of protein A. Doctor says that test is positive for liver cancer. Doctor recommends surgery to collect tissue sample for biopsy. What should you ask the doctor about the blood test?

No cancer Cancer

Liberal criterion No cancer Cancer

Conservative criterion No cancer Cancer