Decision Making Under Uncertainty Let’s stop pretending we know things

Decision Analysis A formal technique for framing and analyzing decisions under uncertainty that have a dynamic component Make decision, then uncertainty revealed, make next decision, … Draws on probability, statistics, economics, psychology Useful for big decisions with a manageable number of alternatives and uncertain elements Like many modeling techniques, process of careful analysis may be as valuable as “results” DA is great tool for helping to structure decision problems DA process leads to useful communications tools describing the problem in a “common language”

Objectives for this Session Help you become an educated consumer of basic decision analyses Use DA to generate broadly applicable fundamental insights regarding decision making

As a consumer, you should be able to Identify opportunities for DA to help frame and analyze tough decisions Play important role in analyzing decision problems by integrating technical analyses with managerial expertise and experience Understand DA principles sufficiently to manage and interact with staff carrying out such analyses

Warning! Decision and risk analysis is a radical concept People, in general, are not comfortable with probabilistic reasoning Most people commonly use point estimates for uncertain quantities and then may carry out a limited 1 or 2 variable sensitivity analysis Everyone will say, “too much thinking and planning required, don’t have time in the real world” but somehow, people have time to revisit the messes they make with “seat of the pants” decision making

Why Important to Model Uncertainty? The world is uncertain Replacing random quantities with averages or single “guesstimates” can be dangerous The Flaw of Averages Allows prediction of distribution of results Not just one predicted number or outcome Sensitivity analysis of outputs to inputs Which inputs really affect the outputs?

Common Decision Making Biases Poor framing – glass ½ full or glass ½ empty Recency effects – the last word Poor probability estimation – uncertain about uncertainty Overconfidence – too certain about uncertainty Escalation phenomena – ignoring sunk cost Association bias – a hammer in search of nails Group think – power in numbers

Random variables (RV) and probability distributions A variable whose value depends on the outcome of an uncertain event Low bid by competing firms Demand for some service next year Number of patients requiring open heart surgery next month at Hospital H Cost of Drug X in December, 2003 Probability of various outcomes determined by probability distribution associated with the RV As modelers, we select appropriate distributions Probability distributions mathematical functions Assign numeric probabilities to uncertain events modeled by the distribution See “Distributions, Simulation and Excel Functions” handout that Doane created.

Discrete Probability Distributions Countable # of outcome values Each possible outcome has an associated probability Expected DemandTotal Probability  A few discrete distributions  Empirical  Binomial – BINOMDIST()  Poisson – POISSON() Expected Value of Discrete RV DistributionReview.xls

Decision Making Elements Although there is a wide variety of contexts in decision making, decision making problems have three main elements: 1. the set of decisions (or strategies) available to the decision maker 2. the set of possible outcomes and the probabilities of these outcomes for all random variables 3. a value model that prescribes results, usually monetary values, for the various combinations of decisions and outcomes. Once these elements are known, the decision maker can find an “optimal” decision. With respect to some decision making objective THEN DO SENSITIVITY ANALYSIS Tornado diagrams

Example – Capacity Planning for Portable Monitoring Devices We need to decide how many monitoring devices to purchase. Here’s our model of demand – a discrete RV. If we’re “short”, we must rent from a supplier at a cost premium. We charge $100/day and incur an estmated cost of $20/day for each monitor we own. How many devices should we purchase?

Decision Analysis Strategy Identify our alternatives Purchase 0, 1, 2, 3, 4, or 5 devices Identify and quantify random variables Demand – we have somehow estimated distribution of daily demand for devices Create payoff matrix for all combinations of alternatives and uncertain outcomes Excel well suited for this Can also graph the risk profile for each alternative Explore “optimal” decision under different objective functions Maximin – maximize the worst possible return (pessimistic) Maximax – maximize the best possible return (optimistic) Expected monetary value – pick the alternative that gives the highest expected return

Daily device cost Revenue # short Total Shortage cost Let’s look at PortMonitoring.xlsPortMonitoring.xls How many devices should we purchase? What does the expected demand suggest we do? Payoff Matrix

Conclusions This comment is in PortMonitoring.xls file.

Effect of Increased Rental Cost

Risk Profiles A risk profile simply lists all possible monetary values and their corresponding probabilities. Risk profiles can be illustrated on a bar chart. There is a bar above each possible monetary value with height proportional to the probability of that value. Making a decision is basically a choice of which risk profile you wish to accept. 3 Devices 5 Devices 4 Devices

The Flaw of Averages A non-linear function of a random variable, evaluated at the average of the random variable, is not the average of the function. F(E(X)) ≠ E(F(X) if F is a nonlinear function When you plug average values into a spreadsheet, you don’t get average outputs unless the model is linear (and most people don’t know if their models are linear or not). Savage, S., 2003, Weapons of Mass Instruction, OR/MS Today, August, pp Practical Interpretation The Math Math Speak

Example of Flaw of Averages This function, probability that the unit is full is NOT a linear function of the birth volume.

Sensitivity Analysis Sensitivity analysis (SA) a big part of Decision Analysis (DA) SA = “What matters in this decision?” which variables might I want to explicit model as uncertain and which ones might I just as well fix to my best guess of their value? On which variables should we focus our attention on either changing their value or predicting their value? No “optimal” SA procedure exists for DA SA can help identify Type III errors - solving the wrong problem

Some SA Techniques Scenarios – base, pessimistic, optimistic How did we do with “scenario planning”? 1-way and 2-way data tables and associated graphs as in the Break Even spreadsheet Tornado diagrams a one variable at a time technique Top Rank –Excel add-in for simple “What if?” Top Rank Risk Analysis or Spreadsheet Simulation direct modeling of uncertainty through probability CrystalBall – sophisticated Excel CrystalBall

Tornado Diagrams Graphical sensitivity analysis technique Create base, low and high value scenarios for each input variable Set all variables at base value “Wiggle” each variable to its low and high values, one at a time. A one-way sensitivity analysis technique Calculate total profit for each scenario Create “tornado diagram” - Excel From “Making Hard Decisions” by Clemen JCHP-BreakEven-Tornado.xls OBMODELS-HCM540-TopRank.XLS

Sensitivity Analysis with TopRank Big bars means high impact

Some of the broadly applicable insights... Explicit incorporation and quantification of risks and uncertainties is often important Be wary of clairvoyant analysts! Several methods for trying to incorporate uncertainty in analysis Quantification of risk is difficult and subject to common human decision biases Humans have hard time with uncertainty It’s important to guard against decision biases Awareness is half the battle It’s OK to say “I DON’T KNOW” Not all information is worth the cost or equally valid Obtaining data for some of these modeling approaches can be difficult probability estimation can be tough historical data may or may not exist