Ming Hsu Everhart Lecture Decisions, Decisions: The Ellsberg Paradox and The Neural Foundations of Decision-Making under Uncertainty Ming Hsu Everhart Lecture

Simple Decisions: Blackjack

Simple Decisions: Blackjack

More Complicated: Investing Stock? Bond? Domestic? Foreign? Diversify Think long-term This is a huge field, and trillians of dollars rests on it. There are a lot of sophisticated models floating around, but for the average investor like you and me, the advice usually basically boils down to uninspired maxims like “diversify” and “think long-term.”

Complicated: Love/Marriage Whether? Who? When? Where? Now some of the most complicated decisions involve highly personal matters like love and marriage. Much has been written on the topic, and many models have been proposed. Not surprisingly, these models are not at all realistic, and make a number of very strong assumptions. Here is one of the more famous: 37% rule. 37% Rule (Mosteller, 1987) “Dozen” Rule (Todd, 1997)

Most of life’s decisions Precise knowledge of probabilities Little knowledge of probabilities Simple Complex Most of life’s decisions One of the things that make the complex decisions complex is: we generally have very little knowledge of the probabilities in these decisions. With something like love, many might even find it vulgar to put assign probabilities and values to decisions. Probably not at Caltech. Love is probably not real to some of you until it can be quantified.

Uncertainty about uncertainty? This might sound at once obvious and strange to you. What does it mean to be uncertain about probabilities, if probabilities ARE our statements about uncertainty. If that is the case, you are in good company. Economists, statisticians, and decision theorists have thought about this question since at least the 1920s.

Ellsberg Paradox In 1961, a thought experiment proposed by Daniel Ellsberg showed very elegantly the following: People behave very differently when they do not know the probabilities versus when they know the probabilities. This became known as the “Ellsberg Paradox.” Incidentally, this is the same Daniel Ellsberg of the Pentagon Papers fame. Prior to that, he was an economist of some promise. 1961

Urn I: Risk 5 Red 5 Blue Most people indifferent between betting on red versus blue

Urn II: Ambiguity 10 - x Red x Blue Most people indifferent between ? ? ? ? ? ? ? ? ? ? Most people indifferent between betting on red versus blue

Choose Between Urns Urn I Urn II (Risk) (Ambiguous) ? ? ? ? ? ? ? ? ? ? Many people prefer betting on Urn I over Urn II. The paradox arises when people are asked to choose from different urns. Do you prefer to bet on RED from Urn I or RED from Urn II? Alternatively, BLUE from Urn I or BLUE from Urn II?

Where Is The Paradox? “…sadly but persistently, having looked into their hearts, found conflict with the axioms and decided … to satisfy their preferences and let the axioms satisfy themselves.” --Daniel Ellsberg, Quarterly Journal of Economics (1961) The paradox lies in the fact that standard decision theory says that one should not behave differently under risk and ambiguity. The Ellsberg Paradox was one of the first challenges to empirical validity of subjective expected utility theory as proposed by LJ Savage.

Ellsberg Paradox Urn I Urn II P(RedI) = P(BlueI) P(RedI) = 0.5 (Risk) Urn II (Ambiguous) P(RedI) = P(BlueI) P(RedI) = 0.5 P(BlueI) = 0.5 P(RedII)=P(BlueII) P(RedII) < 0.5 P(BlueII) < 0.5 ? ? ? ? ? ? ? ? ? ? P(RedI) + P(BlueI) = 1 P(RedII) + P(BlueII) = 1 To see why: consider the following.

Not ambiguity averse Verizon Jennifer or Deutsche Telekom or Angelina Simple Complex Now consider the real world consequences of ambiguity aversion. If people are ambiguity averse, they might choose to invest in companies that are familiar to them (e.g., Verizon) over stocks that are unfamiliar to them (Deutsche Telekom), even in cases where it might be suboptimal to do so. Alternatively, one might choose the familiar, girl-next-door Jennifer over the probably more beautiful, but possible crazy Angelina.

Verizon or Deutsche Telecom? Jokes aside, let me give you some real data. These are the portfolio weights for the mean investor in U.S., Japan, and U.K. The reluctance to hold foreign stocks amounts to a sacrifice in annual percentage return of 1-2% per year. To put it in concrete terms, a ssuming an average unbiased return of 7%, a person with home bias who invests a lump sum at age 25 will end up with only half as much money at age 65 as an investor who is unbiased. French & Poterba, American Economic Review (1991).

Explaining Ambiguity Aversion Like physicists, economists like laws of nature (Law of Demand, Walras’ Law, etc.) Murphy’s Law If anything can go wrong, it will. People consider the worst possible outcome of each action. In 1989, Itzhak Gilboa and David Schmeidler invoked the following. By relaxing a key axiom in Savage’s theory, Gilboa and Schmeidler were able to prove that, when people are faced with ambiguity, they consider the worst possible outcome for each action. In other words, people are slightly paranoid.

Explaining Ambiguity Aversion In other words, people are paranoid.

Explaining Ambiguity Aversion Urn I (Risk) ? Urn II (Ambiguous) P(RedI) = 0.5 P(BlueI) = 0.5 P(RedII|BetRed) = 0 P(BlueII|BetBlue) = 0

What Are We Missing? Unanswered Gilboa & Schmeidler’s model is a model of ambiguity aversion. There are a number of other models of ambiguity aversion. Unanswered Do these models really reflect actual decision-making process? How are the relevant variables interpreted and choices produced? Look in the brain. This is all fine and well as a model of behavior. But does it actually describe reality? This is where we came in.

The Bigger Picture Economics: Human Behavior formal, axiomatic, global. Human Behavior Psychology: intuitive, empirical, local. Neuroscience: biological, circuitry, evolutionary.

- Glimcher and Rustichini. Science (2004) The Bigger Picture Economics: formal, axiomatic, global. Neuroeconomics “A mechanistic, behavioral, and mathematical explanation of choice that transcends [each field separately].” - Glimcher and Rustichini. Science (2004) Human Behavior Psychology: intuitive, empirical, local. This is an area that Caltech is pioneering. It’s something that might sound crazy at first, but when you think about it, is quite obvious: Everyone is studying the same thing, so why not study it together? Now let me tell you why we thought neuroscience can help. I’m going to give you two of the seminal studies that will give you some idea of this. Neuroscience: biological, circuitry, evolutionary.

The Story of Phineas Gage Phineas Gage was a railroad construction foreman in mid-19th C Cavendish, Vermont. On September 13, 1848, he was setting explosive charges in holes drilled into large pieces of rock so they could be broken up and removed. There was an accident. The tamping iron rod ignited the gunpowder and blew clean through Gage’s head. Cavendish, Vermont (September 13, 1848)

The Story of Phineas Gage “…fitful, irreverent, indulging at times in the grossest profanity...” -- Gage’s physician Orbitofrontal Cortex Impulsiveness Poor insight Impaired decision-making Both social and financial Remarkably, Gage regained consciousness after a few minutes, and eventually was able to make a full recovery. However, it soon became clear that Gage was not the same. Whereas before he was responsible, jovial, and hard-working; after the accident, he became…

The second seminal study I will discuss is more recent The second seminal study I will discuss is more recent. Instead of humans, this study is of alert behaving monkeys. The monkey is passively sitting. A stimulus in the form of a geometric shape flashes, some seconds later, a reward in the form of a juice squirt is delivered to the monkey. This whole time, the experimenter is recording the brain activity of the monkey. Here, the monkey is shown a cue that signals a reward will be delivered with probability 1. Fiorillo, Tobler, and Schultz. Science. (2003)

Fiorillo, Tobler, and Schultz. Science. (2003)

Fiorillo, Tobler, and Schultz. Science. (2003)

Tools That We Used Functional Magnetic Brain Lesion Patients Resonance Imaging (fMRI) Brain Lesion Patients

MRI: Magnetization of Tissue Magnetic resonance imaging takes advantage of the fact that different tissue in the brain has different magnetic properties. This is the same sort of MRI that you would take at a hospital for your knee or back.

fMRI: Changes in Magnetization Basal State Activated State fMRI takes advantage of the changes in magnetic properties of brain tissues as different parts of the brain are utilized.

fMRI Time Series Data intensity Statistical Models Time Statistical image (SPM) Click Stop Time voxel time series

Statistical Modeling of fMRI Data b2 + x2 error e b1 x1 = Time Intensity

Random Effects/Hierarchical Models pdf Subj. 2 21 Subj. 1 Subj. 3 1 Subj. 4 Subj. 5 Subj. 6 Distribution of population effect 2Pop Pop

fMRI Experiment 16 Caltech graduate and undergraduate students. 3 Treatments: Card-Deck, Knowledge, Informed Opponents 2 Conditions: Ambiguity, Risk 24 trials per condition per treatment Self-paced trials. No feedback! Hsu, Bhatt, Adolphs, Tranel, and Camerer. Science. (2005)

fMRI Experiment Hsu, Bhatt, Adolphs, Tranel, and Camerer. Science. (2005)

fMRI Experiment Hsu, Bhatt, Adolphs, Tranel, and Camerer. Science. (2005)

Expected Reward Region y - Brain response A(.) - Ambiguity trials R(.) - Risk trials E(.) - Expected value of choices W(.) - Nuisance parameters This is a region that is correlated with expected value of people’s choices. But that’s not all this region is doing.

Lower Activity under Ambiguity % Signal Change The activity of the striatum is also greater under risk than ambiguity.

Lower Activity under Ambiguity % Signal Change These time courses all occur approximately around the time that the subject makes his choice, which is the blue line here. We also know from neuroscience that the striatum is near the output of behavior. Is there another region that is involved in the input?

Region Reacting to Uncertainty Orbitofrontal Cortex y - Brain response A(.) - Ambiguity trials R(.) - Risk trials E(.) - Expected value of choices W(.) - Nuisance parameters N.B. This region does not correlate with expected reward. The orbitofrontal cortex is well situated in the brain for this type of function. It receives rapid, multisensory inputs and is known to evaluate the value of stimuli.

Link Between Brain and Behavior Brain Imaging Data Behavioral Choice Data Stochastic Choice Model

A Signal for Uncertainty? Late Early

Lesion Subjects Orbitofrontal Control

Lesion Experiment Choose between gamble worth 100 points OR 100 Cards 50 Red 50 Black 100 Cards x Red 100-x Black Choose between gamble worth 100 points OR Sure payoffs of 15, 25, 30, 40 and 60 points.

Lesion Patient Behavioral Data Two observations: 1. Choices not random! Both groups clearly respond to incentives. 2. Crucial difference: OFC group less ambiguity and risk averse than Control group.

Estimated Risk and Ambiguity Attitudes Orbitofrontal Lesion Control Lesion Remember this 0.82 Orbitofrontal lesion patients more rational!

Linking Neural, Behavioral, and Lesion Data Brain Imaging Data Imputed value OFC lesion estimate  = 0.82 Behavioral Choice Data Stochastic Choice Model

What have we learned?

One System, Not Two % Signal Change All of our data indicates a single system involved in choice under uncertainty. This has clear implications for decision theory. Ambiguity and risk are part of one system, not too. This makes evolutionary sense.

Reward Value of Ambiguous Gambles Gilboa Schmeidler passes a test.

Signal for Uncertainty

No OFC  No Ambiguity/Risk Aversion Orbitofrontal Cortex

Where are we going?

Neural Circuitry ? We still don’t know the precise interaction between the OFC and the striatum. To do so would require manipulating levels of ambiguity. This is something that Peter Bossaerts is undertaking.

The Brain and Home Bias

Why Ambiguity Averse? “…he was a gambler at heart…[and] assumed that he could always beat the odds.” On Jeffrey Skilling From Bethany McLean and Peter Elkind, Smartest Guys in the Room (2003).

Acknowledgements Colin Camerer Meghana Bhatt Ralph Adolphs Cédric Anen Daniel Tranel Steve Quartz Peter Bossaerts Meghana Bhatt Cédric Anen Shreesh Mysore ELS Committee

END