1. K.M. Corker, Ph.D.Industrial & Systems Engineering Human Factors Experiments ISE 212 Fall 2006 Lecture 2: Evidential Reasoning www.engr.sjsu.edu/kcorker Kevin.Corker@sjsu.edu www.engr.sjsu.edu/kcorker Kevin Corker San Jose State University 9/5/06 Get your facts first, and then you can distort them as much as you please.Get your facts first, and then you can distort them as much as you please. --- Mark Twain (1835 - 1910)Mark Twain

2. K.M. Corker, Ph.D.Industrial & Systems Engineering Summary of Basic Inference The subject is given some evidence and must decide what, if any, is implied by the evidence If P then Q Evidence: P Therefore Q (valid inference) –Affirm the antecedent (modus ponens) (positivist) Evidence: not-Q Therefore not-P (valid inference) (only if P& Q are bivalent, i.e., P can only be true if Q is true –Deny the Consequent (modus tollens) (Not intuitive) But is the basis of critical rationalism E.G.: If the lecture is fascinating, then the students will stay awake: The students are not awake; therefore the lecture is not fascinating

3. K.M. Corker, Ph.D.Industrial & Systems Engineering Critical Rationalism Karl Popper’s method is based in Modus Tollens –Critical Rationalism. Two problems with induction: –psychological problem of finding what you are expecting to find (biases abound) and –logical problem of extending experience to what we have not experienced. He thus found 'common sense' as a scientific justification inadequate method of prediction and statements about what we have not experienced cannot be deemed as 100% 'true'. He also noted that a verificationist approach is less likely to result in new discoveries, as it simply seeks to confirm the beliefs of the scientist.

4. K.M. Corker, Ph.D.Industrial & Systems Engineering Human Decision & Evidential Reasoning Models Normative Models 1) Econometric Models : Expected Value of Decision Outcome People calculate the potential value of each option – Pick the option with the highest expected value Raffle with 10% chance to win $5.00 EV =.10 * $5.00 = $0.50 Simple example Which gamble would you rather play? A: 20% chance of winning $5.00 B: 30% chance of winning $4.50 EV(A) =.20 * $5.00 = $1.00 EV(B) =.30 * $4.50 = $1.35 Expected value suggests you should choose B But Prospect theory – People value a certain gain more than a probable gain with an equal or greater expected value; the opposite is true for losses.–Would you rather win (or lose) $1 and 0% risk or $2 with 50/50 risk?–Take the sure thing?

5. K.M. Corker, Ph.D.Industrial & Systems Engineering Expected Utility The Expected Utility model: EU = Σ ( weighti * utilityi) Expected Utility is a rational model –All choices are transitive –Everything is evaluated relative to a global scale. –But Human Decision & Evidential Reasoning Models

6. K.M. Corker, Ph.D.Industrial & Systems Engineering People treat gains and losses differently – Losses loom larger than gains The same situation feels worse when framed in terms of losses than when framed in terms of gains. –Sunk Cost Bias (Policy Driver) –Culturally Sensitive Kahneman and Tversky: Framing Theory

7. K.M. Corker, Ph.D.Industrial & Systems Engineering Framing & Prospect

8. K.M. Corker, Ph.D.Industrial & Systems Engineering Cue Integration and Estimation Humans are reasonable estimators of mean likelihood. But: –Bad at Non-linear systems extrapolation Rice Grain or Lilly pad pond examples –Bad at estimating tails (representativeness issues) Prior probability (or base-rate frequency) of outcomes often ignored Insensitivity to sample size (Bayesian Inference) Subject to Availability Biases ( Familiarity (salience, recency–driven by experience)

9. K.M. Corker, Ph.D.Industrial & Systems Engineering Example 1x2x3x4x5x6x7x8 = ? 8x7x6x5x4x3x2x1 = ?

10. K.M. Corker, Ph.D.Industrial & Systems Engineering Heuristics & Biases Anchoring Evaluation of simple, conjunctive (and) & disjunctive (or) events: Overestimate conjunctive, underestimate disjunctive Frequency Gambling Overestimate of certainty and reliance on prior (P) Confirmation Bias Seeking confirmatory evidence Automation Bias Overestimate of the quality of machine accuracy and reasoning ability

11. K.M. Corker, Ph.D.Industrial & Systems Engineering Human Uncertainty and Subjective Likelihood Bayesian Decision Making –Combination Method is Bayesian Calculus and uncertainty is captured in the transition probabilities of the world model Fuzzy Logic –Assessment of vagueness or belongingness On a scale from 0 to 1 what degree of belongingness does the incoming information have: –E.g., 4 students are sleeping– on a class-interestingness scale (measured in boredom units) I assign that snooze level a 0.2 of boring. Dempster Shaeffer Theory –Uncertainty is measured by an interval (like a confidence interval in stats) and the span of the interval is the uncertainty. Subjective probability vs. Subjective likelihood –Probability: Assessment of the probability of the observed event (1 in 10 chance of having 1 or more students sleeping) based on an internal probability model informed by experience. –Likelihood: Assessment only based on the present observation without reference to past experience. Naturalistic Decision Making (Recognition Primed Decision making) –Pattern matching using heuristics to select most likely decision based on stored experience with those patterns

12. K.M. Corker, Ph.D.Industrial & Systems Engineering Case Study

13. K.M. Corker, Ph.D.Industrial & Systems Engineering Essence of Case Draft standard, ISO defines Adaptive Cruise Control (ACC) as “an enhancement to conventional cruise control systems which allows the subject to follow a forward vehicle at an appropriate distance by controlling the engine and/or power train and potentially the brake. ACC systems have forward-looking sensors and algorithms that calculate the range and closing rate to a lead vehicle and adjust the cruising speed appropriately. ACC systems are distinguished from Forward Collision Avoidance System (FCAS) in that they do not take evasive action. Warnings may be provided but are not required since ACC’s are described and marketed as convenience systems, not safety systems.

14. K.M. Corker, Ph.D.Industrial & Systems Engineering The plaintiff is a 46 year-old female who sustained serious and permanent injuries as a result of rear-end collision. Her vehicle ACC was set to 70 mph, and 1 second time gap (the minimum setting) Prior to the start of the accident both cars were traveling at 30 m/s, the plaintiff was following at a constant distance of 30 m At time t = 0, the lead vehicle suddenly decelerated hard to avoid hitting a deer reaching a deceleration rate of 9 m/s,s. The ACC system of the plaintiff’s vehicle responded within 200ms to the lead vehicle deceleration with a maximum available deceleration of 3 m/s,s (0.3g).

15. K.M. Corker, Ph.D.Industrial & Systems Engineering The plaintiff initiated emergency braking a t = 2.0 sec, achieving a deceleration of 9 m/s,s The collision occurred at t = 3.2s The relative velocity at impact was 31 mph Safe time to initiate braking (no collision would be ~1.1sec)

16. K.M. Corker, Ph.D.Industrial & Systems Engineering Heuristics Bounded Rationality concept of satisficing, which occur when decision makers stop the search for a solution when the first alternative is found that meets all constraints. Or meets most constraints –Probably not the optimal solution –Fast & frugal heuristics

17. K.M. Corker, Ph.D.Industrial & Systems Engineering Bibliography Fischhoff, B., Slovic, P., Lichtenstein, S., Read, S. & Combs, B. (1978). How safe is safe enough? A psychometric study of attitudes towards technological risks and benefits. Policy Sciences, 8, 127-152. Reprinted in P.Slovic (Ed.), The perception of risk. London: Earthscan, 2001. Kahneman, Daniel, and Amos Tversky (1979) "Prospect Theory: An Analysis of Decision under Risk",Econometrica, XVLII (1979), 263-291. Payne, J.W., Bettman, J.R., & Johnson, E.J. (1988). Adaptive strategy selection in decision making. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 534-552. Klein, G. (1996). Nature of uncertainty in naturalistic decision making. Proceedings of the Human Factors and Ergonomics Society 40th Annual Meeting, 1, 178.