1. Finish: History of JDM Begin: Linear Judgment Models
Psychology 466: Judgment & Decision Making Instructor: John Miyamoto 10/03/2017: Lecture 02-1
Note: This Powerpoint presentation may contain macros that I wrote to help me create the slides. The macros aren’t needed to view the slides. You can disable or delete the macros without any change to the presentation.

2. Outline Brief history of the psychology of decision making
Capacity limitations in human cognitive processes
Four linear judgment models
Why are psychologists interested in linear judgment models?
How can we make decisions based on a linear judgment model?
Psych 466, Miyamoto, Aut ‘17
History of Psychology of Decision Making

3. Cognitive Approach to Judgment & Decision Making (JDM)
Cognitive limitations – limitations on human cognitive capacity affect judgment and decision making
Heuristics and biases movement: – 1990 (approx.)
Reactions to heuristics and biases movement
Evolutionary psychology
Ecological psychology
Naturalistic decision making
Bayesian models of psychological processes
Emotion in decision processes
The Standard Memory Model
Psych 466, Miyamoto, Aut '17

4. Working Memory (WM) Has Severe Capacity Limitations
“Magic number 7 ± 2” – George Miller’s famous paper. WM can only hold a limited number of "chunks" of information.
Remember the sequence of digits: , 85, 8, 2, 67, 14, 80, 66, 77, 65, 5, 18, 4, 7, 26,
Remember the sequence of words: TOP, FROG, SIGN, SLIP,HALL, BOWL, PICK, GRIN, LACK,
Psych 466, Miyamoto, Aut '17
Same Slide but with Additional Comment re the Role of Attention

5. Working Memory (WM) Has Severe Capacity Limitations
“Magic number 7 ± 2” – George Miller’s famous paper. WM can only hold a limited number of "chunks" of information.
When information is complex, people are forced to simplify the reasoning process. Simplifications can lead to distortions.
EXCEPTION: With experience, one can learn to integrate complex information into a small number of "chunks."
Example: Expert chess players can reason about complicated chess problems.
Example: Experienced drivers can understand traffic situations that are actually very complex.
Chunking depends on automatic processes:
Automatic processes  low cognitive demands Controlled processes  high cognitive demands
Psych 466, Miyamoto, Aut '17
Diagram of the Standard Memory Model (from Hastie & Dawes)

6. The Standard Cognitive Model of Human Memory
Sensory Input Buffers
Working Memory
Central Executive
Phonological Buffer
Goal Stack
Visuospatial Buffer
Long-Term Memory
Hastie & Dawes Fig. 1.1
The Standard Cognitive Model of Human Memory
Sensory Registers
Psych 466, Miyamoto, Aut '17

7. Hastie & Dawes Fig. 1.1
Sensory Input Buffers
Working Memory
Central Executive
Phonological Buffer
Goal Stack
Visuospatial Buffer
Long-Term Memory
Sensory registers retain the sensory information for very brief periods of time.
Psych 466, Miyamoto, Aut '17
Working Memory

8. Working memory (WM) holds a limited amount of information.
Hastie & Dawes Fig. 1.1
Sensory Input Buffers
Working Memory
Central Executive
Phonological Buffer
Goal Stack
Visuospatial Buffer
Long-Term Memory
Working memory (WM) holds a limited amount of information.
Thoughts are actively manipulated in WM.
Capacity limit for unrelated pieces of information is about 7 ± 2 pieces.
Information in WM lasts less than 20 seconds if it is not actively processed.
Long-Term Memory
Psych 466, Miyamoto, Aut '17

9. Hastie & Dawes Fig. 1.1
Sensory Input Buffers
Working Memory
Central Executive
Phonological Buffer
Goal Stack
Visuospatial Buffer
Long-Term Memory
Long-term memory (LTM) retains information over longer periods of time (days, months, even years). LTM interacts with WM.
Information is retrieved from LTM to be processed in WM.
General Hypothesis of Cognitive Research
Psych 466, Miyamoto, Aut '17

10. General Hypothesis of Cognitive Research
Hastie & Dawes Fig. 1.1
Sensory Input Buffers
Working Memory
Central Executive
Phonological Buffer
Goal Stack
Visuospatial Buffer
Long-Term Memory
General Hypothesis of Cognitive Research
Limitations in working memory impose limitations on human ability to engage in complex reasoning.
Decision making requires complex reasoning.
Psych 466, Miyamoto, Aut '17
Summary So Far: Where Are We in this Lecture?

11. Where Are We in the Lecture?
Rational agent model ignores issue of cognitive limitations.
Normative and prescriptive decision models require complex representations and processing.
Cognitive limitations cause us to simplify decisions, and this can produce errors.
NEXT: How to Deal with Cognitive Complexity
Intuitive judgment versus statistical models
Brunswik’s Lens Model of Human Judgment
Linear models applied to making better choices
Applications to clinical judgment
Intuitive Judgment vs Actuarial Judgment
Psych 466, Miyamoto, Aut '17

12. Intuitive Judgment versus Acturial Judgment
Combine complex information in your head
Make decision based on gut feeling
Actuarial judgment (a.k.a. statistical model or linear model)
Base decisions on a statistical decision rule.
Intellectual warfare between cognitive psychology and clinical psychology. (Especially in 1950's 's).
Statistical Models Outperform Human Judges
Psych 466, Miyamoto, Aut '17

13. Examples of Judgment Problems
We will only consider decisions to which intuitive judgment and actuarial judgment (statistical methods) both apply
E.g., Clinicians attempt to identify patients with progressive brain dysfunction.
Data = intellectual test results
Experienced clinicians achieved 58% correct detection of new cases.
Statistical model achieved 83% correct detection of new cases.
E.g., Bank loan officer must decide which loan applications are “good risks” and which are “bad risks.”
E.g., College admissions committee must decide which high school applicants will do well at the college level.
Critique of Clinical Judgment – What Is It?
Psych 466, Miyamoto, Aut '17

14. Critique of Clinical Judgment
Clinical insight – does it exist?
Clinical judgment – what are its strengths?
Clinical judgment – what are its weaknesses?
Accusation: Belief in the efficacy of intuitive clinical judgment is a cognitive conceit.
Psych 466, Miyamoto, Aut '17
General Finding: Stat Models Outperform Human Judges

15. General Finding: Stat Models Outperform Human Judges
Statistical models almost always outperform the human judges on clearly defined decision tasks.
Human cognitive processes are good at noticing particular pieces of information.
Does my friend look happy? Sad? Stressed? Irritated?
Is the patient nervous? Defensive? Exhibitionistic?
Human cognitive processes are not good at integrating multiple pieces of information.
Can I predict how my friend will feel about a surprise party?
Can the clinician predict how the patient will progress after 4 months of therapy?
Implications of this Topic
Psych 466, Miyamoto, Aut '17

16. Implications of this Topic
We can improve human decisions by stressing what humans are good at:
... noticing what are important issues that are relevant to a decision;
... evaluating how good or bad something might be on a specific dimension;
while avoiding what we are not good at:
... combining complex information in our heads.
Know thyself → Make better decisions
* Brunswik's lens model is an attempt to characterize the problem of human judgment.
Brunswik's Lens Model
Psych 466, Miyamoto, Aut '17

17. The Lens Model of Egon Brunswik
Figure 3.1 of Hastie & Dawes.
To-be-judged criterion = the thing you are trying to predict.
Judgment = the “judge’s” prediction (you are the judge)
Cues = things you can observe about the criterion
The lens model is a conceptualization of the structure of typical judgment problems.
Psych 466, Miyamoto, Aut '17
Examples of Jdmt Problems: Explain Criterion, Cues & Judgment

18. Brief Digression: Definition of Holistic Judgment
Psych 466, Miyamoto, Aut '17
Brief Digression: Definition of Holistic Judgment

19. Holistic Judgment
Holistic judgment - judgment of a complex trait from multiple cues by means of a single intuitive judgment as opposed to a calculation based on a formula.
Example of Holistic Judgment
Psych 466, Miyamoto, Aut '17

20. Example of Holistic Judgment
Suppose a clinician has to decide whether a patient is suicidal and should be hospitalized.
He or she considers patient’s appearance, what patient says, the patient’s background and record, previous interactions, etc.
Eventually, the clinician makes a decision based on an intuitive integration of all this information
Note: Holistic judgment may include stages in which the judge considers component features of the decision.
What makes it a holistic judgment is that the ultimate evaluation is made by the judge through an intuitive integration of all of the information about the decision.
Digression on the Meaning of “Holism”
Psych 466, Miyamoto, Aut '17

21. Terminology: Holism and Analytical Strategies
Holistic judgments are sometimes called global judgments to distinguish them from judgments that evaluate the separate cues “Holism” is sometimes written as “wholism.”
Holistic judgment strategies are contrasted with analytical judgment strategies. Analytical judgment strategy: Break the decision into component parts; use an explicit calculation to combine these parts.
Claims Regarding Holistic & Analytical Judgment Strategies
Psych 466, Miyamoto, Aut '17

22. Claims Regarding Holistic & Analytical Judgment Strategies
Claim: An analytical judgment strategy will generally produce better predictions than a holistic judgment strategy.
E.g., clinical judgments are more often correct if they are made analytically than if they are made holistically.
E.g., your own predictions of what will happen in sports events, on the stock market, and predictions about social behavior will be more often correct if they are made analytically than if they are made holistically.
Related Claim: An analytic judgment strategy is better than an intuitive judgment strategy because it provides a better way to combine complex information.
Analytical versus Intuitive Judgment Strategy (cont.)
Psych 466, Miyamoto, Aut '17

23. Do Analytical Judgment Strategies Exclude Human Intuition?
NO!
A human judge is still needed to ...
... decide what are the important issues in a decision;
... help create an analysis of the decision structure;
... judge how good or bad an outcome would be on a particular dimension,
e.g., social impact of a decision, health impact of a decision, political impact of a decision, etc.
Four Linear Judgment Models
Psych 466, Miyamoto, Aut '17

24. Four Linear Judgment Models
Using multiple regression on objective data for which the true state is known. Apply this model to new data for which the true state is not known.
Using multiple regression on judgment data where the true state is not known.
SMART (Simple Multi-Attribute Rating Technique)
Unit Weighting Model
Except for Model 4 (unit weighting), the models do NOT describe everyday judgment.
Some natural models are similar to these models.
Using any of these models can improve human performance.
Psych 466, Miyamoto, Aut '17
Reminder: Brunswik's Lens Model

25. Four Linear Judgment Models
Using multiple regression on objective data for which the true state is known. Apply this model to new data for which the true state is not known.
Using multiple regression on judgment data where the true state is not known.
SMART (Simple Multi-Attribute Rating Technique)
Unit Weighting Model
Assignment 1: Explain how to use a linear judgment procedure to choose 1 UW course to take.
Minimize issues of major or minor requirements.
Compare intuitive judgment and linear judgment models.
On Assignment 1, you only need to explain 1 of 4 methods.
Psych 466, Miyamoto, Aut '17
Reminder: Brunswik's Lens Model

26. Next: Explain These Four Methods on a Concrete Judgment Problem
Concrete Problem: Predicting college performance (GPA) based on information in a college application.
Predictions based on a linear model – how to produce them
Predictions based on a model of the judge – how to produce them
Predictions based on SMART method
Predictions based on unit weights – how to produce them
Assignment 1: Pick one of these four methods. Describe this method and then discuss its strengths and weaknesses relative to other methods (including intuitive judgment.
Continue the Intro to Baron's College Admission Problem
Psych 466, Miyamoto, Aut '17

27. Judgment Data Used in Baron's Chapter 20
* Baron's data are stored in <\P466\nts\baron.quant.jdmt.r-code.docm>.
COL = college GPA. This is the criterion This is what the judge wants to predict.
SAT = SAT score; REC = judge's rating of the recommendation; ESS = judge's rating of the student's essay; GPA = high school GPA These are the cues.
Comment re Qualitative Variables
Psych 466, Miyamoto, Aut '17

28. Baron's College GPA Judgment Problem
Using the terminology of the lens model, the GPA judgment problem looks like this:
Terminology
Example
Criterion (what we want to predict)
COL = College GPA of high school student
Cues (these are things we know)
GPA = High school GPA
SAT = SAT test scores
REC = recommendations (converted to ratings)
ESS = essay quality (converted to a rating)
Judgment (this is our prediction)
PRE = Estimate (guess) of student’s future college GPA
* Baron's data are stored in <\P466\nts\baron.quant.jdmt.r-code.docm>.
Table Showing Data & Variables for the College Admission Example
Psych 466, Miyamoto, Aut '17

29. Qualitative Variables Must Be Converted To Quantitative Variables
SAT, GPA are already quantitative.
REC = strength of recommending letters is qualitative; convert to quant measure by having humans rate the letters for how positive they are.
ESS = quality of applicant’s essay is qualitative; convert to quant measure by having humans rate the essay for how good it is.
Psych 466, Miyamoto, Aut '17
Four Methods for Predicting Future Cases

30. Four Ways to Compute a Statistical Prediction Model
Next
Method 1: Multiple regression applied to existing data Called a “proper linear model”
Method 2: Multiple regression applied to a judge’s predictions Called a “model of the judge”
Method 3: SMART Method with "importance" weights Called the SMART method or importance weighting method
Method 4: Unit weighting model Called the unit weighting model or unit weighting method
Multiple Linear Regression
Psych 466, Miyamoto, Aut '17

31. Method 1: Multiple Linear Regression
Multiple linear regression is a statistical method for finding a formula that predicts the criterion from a set of data.
PRE = statistical prediction (predicted college GPA) produced by Method 1
COL is the criterion
These are the cues
Prediction Equation from the Multiple Regression
Psych 466, Miyamoto, Aut '17

32. Prediction Equation with 4 Predictor Variables
COL = the criterion = college GPA = to-be-predicted quantity
SAT, REC, ESS, GPA are the cues (predictor variables)
See ‘e:\p466\nts\baron.quant.jdmt.r-code.docm’ for R-code that computes the prediction from the linear regression.
Prediction Equation
PRE = ·SAT ·REC ·ESS ·GPA  5.161
The regression weights for the prediction equation are underlined above:
A statistics program can compute the regression weights based on the data in the table.
Example of a Regression Equation
Psych 466, Miyamoto, Aut '17

33. Prediction Equation with 4 Predictor Variables
COL = the criterion = to-be-predicted quantity
SAT, REC, ESS, GPA are the cues (predictor variables)
Prediction Equation
PRE = ·SAT ·REC ·ESS ·GPA  5.161
Example: If a high school student has SAT = 1200, REC = 3.7, ESS = 3.9, GPA = 3.2, then we predict
PRE = ·(1200) ·(3.7) ·(3.9) ·(3.2) – 5.161 =
How to Use the Multiple Regression Model to Predict New Cases
Psych 466, Miyamoto, Aut '17

34. How We Use the Linear Regression Model
See ‘e:\p466\nts\baron.quant.jdmt.r-code.doc’ for R-code that computes the regression equation.
Step 1: Apply multiple regression calculation to data for which the value of the criterion (COL) is known This produces the prediction equation.
Step 2: Use the prediction equation to predict the criterion (COL) for new cases where the value of the criterion is NOT known.
Linear Model Outperforms the Human Judge
Psych 466, Miyamoto, Aut '17

35. Linear Model Outperforms Human Judges
In Baron’s college GPA example, the statistical model makes more accurate predictions of college GPA than do expert human judges.
Same finding on many other examples (see Dawes, Faust & Meehl paper)
Is this surprising? Is it interesting from the standpoint of psychology? Who cares? Or who might care?
Judges think that they make their judgments by means of a complex, interactive, nonlinear evaluation of the cues.
Whether or not the judge's introspections are veridical, the results of these studies show that a much simpler combination of the cues can produce better predictions than the processes used by human judges.
Return to Outline of Models
Psych 466, Miyamoto, Aut '17

36. Four Ways to Compute a Statistical Prediction Model
Method 1: Multiple regression applied to existing data Called a “proper linear model”
Method 2: Multiple regression applied to a judge’s predictions Called a “model of the judge”
Method 3: SMART Method with "importance" weights Called the SMART method or importance weighting method
Method 4: Unit weighting model Called the unit weighting model or unit weighting method
Next
Model of the Judge
Psych 466, Miyamoto, Aut '17

37. Method 2: Model of the Judge
Researcher has available the scores for SAT, REC, ESS and GPA.
The values of the criterion (COL) are NOT available.
Researcher asks the judge to make intuitive, global predictions for these cases. This produces the column labeled "JUD” (next slide).
Not Available
Not Available
Same Slide Except JUD Column Added to Table
Psych 466, Miyamoto, Aut '17

38. Method 2: Model of the Judge
Researcher has available the scores for SAT, REC, ESS and GPA.
The values of the criterion (COL) are NOT available.
Researcher asks the judge to make intuitive, global predictions for these cases. This produces the column labeled "JUD."
Not Available
Not Available
Regression Equation for the Model of the Judge
Psych 466, Miyamoto, Aut '17

39. Method 2: Model of the Judge
It is an accident that in this example, JUD and MUD are identical
Compute a regression model that predicts JUD (Model of the jUDge or MUD).
Example: “Policy Capturing”.
Not Available
Not Available
See ‘e:\p466\nts\baron.quant.jdmt.r-code.docm’ for R-code that computes the prediction from the model of the judge.
Example:
MUD = (–5x10–18)·SAT ·REC ·ESS ·GPA  4.8
Use the MUD model to predict college GPA (COL) for these cases or future cases.
Pstch 466, Miyamoto, Aut '17
Discussion of Model of the Judge

40. Discussion of Case 2: Model of the Judge (MUD)
Empirical findings for Case 2 are the same as for Case 1
MUD more accurate than the intuitive judgments of the judge.
Judges think that they make their judgments by means of a complex, interactive, nonlinear evaluation of the cues.
Whether or not this is really true, studies show that a much simpler combination of the cues produces better predictions.
We don't need to know the value of the criterion (COL) in order to find a statistical formula (prediction equation) that can outperform the judge. .
Why Does MUD Outperform the Judge?
Psych 466, Miyamoto, Aut '17

41. Why Does MUD Outperform the Judge?
The model of the judge (MUD) is a model of the judge’s decisions, not of the criterion (reality).
The MUD outperforms the judge because the statistical formula is consistent – it treats each case by the same formula.
A human judge has random variations in his or her judgment. Random variations increase the inaccuracy (error) of the judge's predictions.
A human judge sees various “special circumstances” (anecdotal memories) that suggest that a specific case should be judged differently from the standard procedure.
Psych 466, Miyamoto, Aut '17
Four Methods – SMART Method is Next

42. Four Ways to Compute a Statistical Prediction Model
Method 1: Multiple regression applied to existing data. Called a “proper linear model”
Method 2: Multiple regression applied to a judge’s predictions Called a “model of the judge”
Method 3: SMART Method with "importance" weights Called the SMART method or importance weighting method
Method 4: Unit weighting model Called the “unit weighting model or unit weighting method”
Next
SMART Method
Psych 466, Miyamoto, Aut '17

43. Method 3: The SMART Method
Simple Multi-Attribute Rating Technique (SMART)
A.k.a. Importance Weighting Model
This technique lets us rank order the individual cases (high school students). It does not produce a predicted GPA of each student.
A rank order tells us who is predicted to do better or worse. This is enough to guide our selection of students.
The SMART method can be modified to actually predict college GPA (COL), but we won’t go into this complication because it is not needed in a choice situation.
Step by Step Explanation of SMART Method
Psych 466, Miyamoto, Aut '17

44. Method 3: The SMART Method
Step 1: Convert the predictors, SAT, REC, ESS and GPA to z-scores.
Step 2: The judge decides what is the relative importance of these variables, e.g., SAT is twice as important as ESS, GPA is 50% more important than SAT, etc.
Step 3: Create a prediction equation that reflects the judge’s judgment as to the relative importance of the predictor variables.
Example of Prediction of MAUT Model = Predicted Score = 2·Z.sat + 3·Z.rec + 1·z.ess + 2·Z.gpa
Tables Showing Original Data, Z-Scores, and Prediction
Psych 466, Miyamoto, Aut '17

45. Method 3: The SMART Method
Z-Scores
R-code for the table to the right is in ‘e:\p466\nts\baron.quant.jdmt.r-code.doc’
Table on the left shows the initial data.
Table on the right shows the z-scores for the predictor variables, and the predicted rating for each student.
Predicted Score = 2·Z.sat + 3·Z.rec + 1·z.ess + 2·Z.gpa
The “Predicted” column tells you who is predicted to do better or worse. It does not tell you the predicted GPA.
Psych 466, Miyamoto, Aut '17
Example: Computing the Predicted Score for One Student

46. Method 3: The SMART Method
Z-Scores
R-code for the table to the right is in ‘e:\p466\nts\baron.quant.jdmt.r-code.doc’
Example of Prediction of MAUT Model = Predicted Score = 2·Z.sat + 3·Z.rec + 1·z.ess + 2·Z.gpa
Example for 1st student: (2.39) + 3(.32) + 1(.41) + 2(1.58) = 9.31
Comment on How to Convert Prediction to a Predicted College GPA
Psych 466, Miyamoto, Aut '17

47. Method 3: The SMART Method
Z-Scores
R-code for the table to the right is in ‘e:\p466\nts\baron.quant.jdmt.r-code.doc’
To convert the “Predicted” column to an predicted college GPA, you need to furnish a guess as to the mean and variance of college GPA’s (unnecessary for your assignment).
Producing a predicted college GPA by this method requires a few math tricks. They are not worth discussing in Psych 466.
Findings for the SMART Method
Psych 466, Miyamoto, Aut '17

48. Findings for Method 3: SMART Method
Findings for Method 3 are the same as for Method 2 (Model of the Judge)
SMART method is more accurate than the intuitive judgments.
The predicted ratings that are computed by the SMART method correlate better with actual results than do the judges intuitive predictions.
More Discussion of the SMART Method
Psych 466, Miyamoto, Aut '17

49. Discussion of Method 3: SMART Method
The SMART method does not require that we know the value of the criterion (COL) for a set of known cases.
We don't need a complicated calculation of the optimal regression weights (Methods 1 & 2 require this calculation).
The SMART Method is better than intuitive judgment because ….
It approximately captures the relative importance of the predictor variables, and ….
It is consistent – it is not bothered by shifts of attention, fatigue, distractions, anecdotal memories, etc.
Four Methods – Next We Discuss Unit Weighting Model
Psych 466, Miyamoto, Aut '17

50. Four Ways to Compute a Statistical Prediction Model
Method 1: Multiple regression applied to existing data. Called a “proper linear model”
Method 2: Multiple regression applied to a judge’s predictions Called a “model of the judge”
Method 3: SMART Method with "importance" weights Called the SMART method or importance weighting method
Method 4: Unit weighting model Called the “unit weighting model or unit weighting method”
Next
Unit Weighting Method
Psych 466, Miyamoto, Aut '17

51. Reminder: What Are the Weights?
Method 1: Proper Linear Model PRE = ·SAT ·REC ·ESS ·GPA  5.161
Method 2: Prediction Equation for the Judge’s Predictions MUD = (–5x10–18)·SAT ·REC ·ESS ·GPA  4.8
Method 3: Predicted Scores Based on Importance Weights: Predicted Score = 2·Z.sat + 3·Z.rec + 1·z.ess + 2·Z.gpa
Method 4: Unit Weighting Model:
Predicted Score = 1·Z.sat + 1·Z.rec + 1·z.ess + 1·Z.gpa
The weights are the numbers that are assigned to the predictor variables.
Further Explanation of the Unit Weighting Model
Psych 466, Miyamoto, Aut '17

52. Method 4: Unit Weighting Model
Z-Scores
R-code for the table to the right is in ‘e:\p466\nts\baron.quant.jdmt.r-code.doc’
Table on the right shows the z-scores for the predictor variables, and the predicted rating for each student.
Example for Case 1: (2.39) + 1(.32) + 1(.41) + 1(1.58) = 4.70
Findings for the Unit Weighting Model
Psych 466, Miyamoto, Aut '17

53. Findings for Method 4: Unit Weighting Model
Unit weighting model more accurate than the intuitive judgments of the judge.
Judges think that they make their judgments by means of a complex, interactive, nonlinear evaluation of the cues. Whether or not this is really true, studies show that a much simpler combination of the cues produces better predictions.
We don't even need to calculate optimal regression weights.
All we need to know is which cues are positively related to the criterion and which are negatively related to the criterion.
Discussion of the Unit Weighting Model (cont.)
Psych 466, Miyamoto, Aut '17

54. Discussion of Method 4: Unit Weighting Model (cont.)
Unit weighting model outperforms human judge because it is consistent - it treats each case by the same formula.
A human judge has all sorts of random variations in his or her judgment. These random variations simply increase the inaccuracy (error) of the judge's predictions.
A human judge may be influenced by anecdotal memories. The unit weighting model is not influenced by these memories.
Comment: Results for unit weighting model show that the critical weakness of the human judge is inconsistency, not the inability to produce optimal weights.
Table: Pros and Cons of Prediction Methods
Psych 466, Miyamoto, Aut '17

55. Only if you want to make a quantitative prediction
This table is Table 3 in ‘e:\p466\hnd.02-2a.p466.a13.docm’. The table was copied to an image and pasted into these slides.
Repeat this Table Without the Red Rectangles
Psych 466, Miyamoto, Aut '17

56. This table is Table 3 in ‘e:\p466\hnd. 02-2a. p466. a13. docm’
This table is Table 3 in ‘e:\p466\hnd.02-2a.p466.a13.docm’. The table was copied to an image and pasted into these slides.
Conclusions
Psych 466, Miyamoto, Aut '17

57. Tuesday, 3 October, 2017: The Lecture Ended Here
Psych 466,, Miyamoto, Aut '17

58. What We Have Learned from the Study of Linear Judgment Models
Human judges typically believe that they use complex judgment processes to make judgments from complex cues. (This may be true.)
The part of the human judgment process that validly predicts the criterion is well modeled by a simple linear model.
We don't need to use optimal regression models to outperform human judges.
We don't need to know the value of the criterion in order to create a model that outperforms human judges.
Evidence: Model of the judge, unit weighting and importance weighting models outperform the human judge.
Note: These methods work only because the human judge has some valid knowledge of the relationship between the cues and the criterion.
Sample Size Affects the Predictive Accuracy of Multiple Regression & MUD
Psych 466, Miyamoto, Aut '17

59. Sample Size Affects the Accuracy of Multiple Regression Model and Model of the Judge
This point is emphasized by Gigerenzer: If you are using a multiple regression model or the model of the judge, the predicted accuracy of the model is bad if the sample size is too small.
If the sample size is small, then the regression weights will tend to be inaccurate. (Technically, the variance of the regression weights greatly increases as the sample size gets smaller.)
If the sample size is small, the unit weighting model can be more accurate than multiple regression model or the model of the judge. (This point is emphasized by Gigerenzer.)
Psych 466, Miyamoto, Aut '17
Why Do Statistical Model Outperform Humans?

60. Why Do Statistical Models Outperform Human Judges?
Human judgment is affected by internal random variation; statistical model is not.
Human judgment is affected by vivid individual cases (anecdotes); statistical model is not.
Speculation: Human judge tries to fit information into a story; statistical model ignores story; it just adds up the evidence But is the human preference for stories bad?
General Discussion of Linear Judgment Models - END
Psych 466, Miyamoto, Aut '17

61. General Discussion of Linear Models
Why aren’t linear judgment models used more widely in practical decision making?
College or graduate admissions
NIH or NSF grant review committees
Political decisions like where to locate a prison; where to locate a homeless shelter;
The Denver bullet study
Are linear judgment methods dehumanizing, e.g., when choosing who will be admitted to a college?
END
Psych 466, Miyamoto, Aut '17

62. Set Up for Instructor
Turn off your cell phone. Close web browsers if they are not needed.
Classroom Support Services (CSS), 35 Kane Hall,
If the display is odd, try setting your resolution to 1024 by 768
Run Powerpoint. For most reliable start up:
Start laptop & projector before connecting them together
If necessary, reboot the laptop
Psych 466, Miyamoto, Aut ‘17