1. Doing better: Some practical advice

2. How to see past our biases and get at the truth?
Data-driven, quantitative risk modeling and decision-making

3. Ten barriers to making good unaided decisions
Plunging in
Frame blindness
Lack of frame control
Overconfidence in your judgment
Shortsighted shortcuts
Shooting from the hip
Group failure
Fooling yourself about feedback
Not keeping track
Failure to audit your decision process

4. Plunging in Symptoms Cures Solving the wrong problem
Not solving the right one
Not solving anything; analysis-paralysis
Omitting key objectives, alternatives, consequences
Solving it badly (missing a clearly better solution)
Cures
Before starting to make decisions, identify: What is the crux of the problem?
Step back: Answer, How should decisions like this one be made? (“meta-decision”)
Control framing deliberately and well.
Choose frames to improve decisions.
What to focus on, what to ignore?

5. Plunging in Symptoms Cures Solving wrong problem Not solving right one
Not solving anything; analysis-paralysis
Omitting key objectives, alternatives, consequences
Solving it badly (missing a clearly better solution)
Cures
Before starting to make decisions, identify: What is the crux of the problem?
Step back: Answer, How should decisions like this one be made? (“meta-decision”)
Framing =
decision boundaries
reference points (“success”)
measures
guiding metaphors
Examples:
sunk-cost fallacy
gain vs. loss frames
endowment effect
house money effect

6. Examples of frame blindness
Surgery vs. radiation therapy
Comparing outcomes to poor reference groups
Walk 2 blocks to save $30 on:
(a) A new $70 watch; or
(b) A new $800 computer

7. Improving decision-making
Select frames deliberately and well
Frames drive decisions!
Collect and use information effectively
What information should you seek?
Ask disconfirming questions. Experiment!
Fault trees, scenarios, prospective hindsight
How should you express and assess beliefs?
Use confidence intervals, calibration
Good group decisions need productive conflict
Use objective models to improve decisions
Learn effectively from experience
Avoid:
fast but stupid decision rules (e.g., screening & ranking)
substituting your confident judgments for factual data
confirmation bias
information overload, pursuit of useless information
uncritical trust in traditional rules of thumb
recency, saliency, anchoring & availability biases

8. Collect and use information effectively
What information should you seek?
What data are relevant?  Wason selection
Ask disconfirming questions. Experiment!
Fault trees
Scenarios
prospective hindsight
How should you express and assess beliefs?
Use confidence intervals, calibration
Good group decisions need productive conflict

9. Use objective (data-driven) quantitative models to improve decisions
Creating models
Validating models
Critiquing models: How good is performance?
Face validity
Descriptive validity
Predictive validity
Implementing models
Refining and updating

10. Why bother to use objective models?
They work! +
Framing effects distort our perceptions and choices
Allais paradox (1953)
Repeated gambles
Use of relative instead of absolute changes
Excessive loss aversion
Reference dependence
Irrational disappointment, regret

11. How to do it? Creating models Validating models Critiquing models
Learning from data & knowledge
Predicting, assessing, adapting
Refining and updating
Validating models
Critiquing models
Implementing models
Using models effectively

12. Learning more effectively
Avoid biases that impair learning
confirmation bias
hindsight bias. (We did not “know it all along”!)
illusion of control
self-serving attribution biases
Objectively attribute effects to their causes
Value information correctly
Be willing to explore and experiment
Know when to stop investigating, and decide!
Flexible, adaptive learning

13. Obstacles to learning: Rationalizaton biases
Misremember what really happened, to make ourselves look better
Blame failures on others, or unforeseeable events
Self-serving biases and explanations
“Success is due to skill, failure to bad luck”
Example: Predictable cost overruns
Re-interpret original predictions to make them sound better (in light of the facts) than they were
Change preferences to make failure ok
“Sour grapes”
Cures
Specify goals, criteria, extent of control, in advance
Debrief after project
Review successes and failures periodically.
For repeated decisions, use statistical analysis to quantify contributions of chance and choice

14. Improving feedback Close gaps in feedback loops
Be aware of treatment/placebo effects
Correct for random effects (“noise”)
Keep track of relevant information
Use information: Don’t ignore useful feedback!
Design experiments to learn how to do better

15. Ten barriers to making good unaided decisions
Plunging in
Frame blindness
Lack of frame control
Overconfidence in your judgment
Shortsighted shortcuts
Shooting from the hip
Group failure
Fooling yourself about feedback
Not keeping track
Failure to audit your decision process