Sensitivity Analysis and Uncertainty H. Scott Matthews /73-359/ Lecture 6

Admin zHW 2 Due today yPlease don’t send s/come by an hour before class and expect answers zCases: great discussion Monday (from Chris) yWriteups good, will grade “best of X” zFriday recitation: microeconomics review zWhen to schedule take-home final? zHow to CDs? yAll CEE cluster machines pre-installed.

Problem of Unknown Numbers  If we need a piece of data, we can:  Look it up in a reference source  Collect number through survey/investigation  Guess it ourselves  Get experts to help you guess it  Often only ‘ballpark’, ‘back of the envelope’ or ‘order of magnitude needed  Situations when actual number is unavailable or where rough estimates are good enough  E.g. 100s, 1000s, … (10 2, 10 3, etc.)  Source: Mosteller handout

Uncertainty zInvestment planning and benefit/cost analysis is fraught with uncertainties yforecasts of future are highly uncertain yapplications often made to preliminary designs ydata is often unavailable zStatistics has confidence intervals – economists need them, too.

Definition: “Base Case” zGenerally uses single values and our ‘best guesses’ zSensitivity Analysis acknowledges uncertainty exists zIncorporate variables instead of constant assumptions zIf our ‘Net Benefits’ remain positive over a wide range of reasonable assumptions, then robust results

How many variables? zChoosing ‘variables’ instead of ‘constants’ for all parameters is likely to make model unsolvable zPartial sens. Analysis - change only 1  Equivalent of  y/  x yDo for the most ‘critical’ assumptions yCan use this to find ‘break-evens’

Best and Worst-Case Analysis zAnalogous to “upper and lower bounds” used in estimation problems zDoes any combination of inputs reverse the sign of our answer? yIf so, are those inputs reasonable? yE.g. using very conservative ests. yMight want NB > 0, but know when NB < 0 ySimilar to ‘breakeven analysis’

Question 2.4 from Boardman z3 projects being considered R, F, W yRecreational, forest preserve, wilderness yWhich should be selected?

Question 2.4 Project “R with Road” has highest NB

Question 2.4 w/ uncertainty zWhat if we are told that Benefits/Costs of each project are uncertain by, for example plus or minus 10%? ye.g. instead of Project R having benefits of $10 million, could be as low as $9 million or as high as $11 million yRepeat for all project combinations zNow which project is ‘best’?

Question 2.4 w/ uncertainty Best Case: R w/Road (same) Worst Case: Road Only General Case: Could be several But difficult to determine that from this chart - can we do better?

Using error/uncertainty bars zShow ‘original’ point as well as range of uncertainty associated with point yRange could be fixed number, percentage, standard deviation, other zExcel tutorial available at: yhttp://phoenix.phys.clemson.edu/tutorials/ex cel/advgraph.html ySee today’s spreadsheet on home page yGraphs original points, and min/max deviations from that as error bars…

Error bar result Easier to see results - imagine moving ruler up and down the axis There is a range from about $3 to $4 million with 3+ options possible

An Easy Visual Clue - Slider Bars zUse built-in excel functions to visually see effects of incremental changes in variables y“Scroll bars” (form toolbar) yTutorial at: ollbar/scrollbar.viewlet/scrollbar_viewlet_swf.html ollbar/scrollbar.viewlet/scrollbar_viewlet_swf.html yLet’s look at our old TV estimation problem xTool gives easy, visual aid for how sensitive parameters are (but not very quantitative)

Case: Photo-sensors for lighting zSave electricity by installing sensors in areas where natural light exists ySensors ‘see’ light, only turn light fixtures on when needed zMAIN Posner 2nd floor hallway uses watt fluorescent bulbs for 53 fixtures zHow could we make a model to determine whether this makes sense? yAssume only one year time frame

Photo-sensors for lighting zAssume we only care about ‘one year project’ zCosts = Labor cost, installation cost, electricity costs, etc. yAssume each bulb costs $6 zBenefits = ? How should we set up model? yAssume equal, set up as ‘show minimum cost’ option zCase 1 ‘Status quo’: assume lights used as is yOn all the time, bulbs last 10,000 hours ~ burn out once per year) zCase 2 ‘PS’: pay to install sensors now, bulbs off between 1/3 and 1/2 of time

Lighting Case Study - Status Quo zCosts(SQ) - lights on all the time yLabor cost: cost of replacing used bulbs x“How many CMU facilities employees does it take to change a light bulb?” - and how long does it take? xAssume labor cost = $35/hr, 15 mins/bulb x26.5 hours to change all bulbs each year, for a total labor cost of $927.50! Also, bulb cost $636/yr yElectricity: 106*15W ~ 14,000 kWh/yr (on 24-7) xCost varies from cents/kWh ~ $350-$1045 xCost Replacing bulbs is same ‘order of magnitude’ as the electricity! (Total range [$1,911 - $2,608])

Lighting Case Study - PS sensors zCosts(PS) - probably ‘off’ 1/3 - 1/2 of time yLabor cost: cost of installing sensors = ‘unknown’ yLabor cost: cost of installing new bulbs xCould assume 1/2 - 2/3 of bulbs changed per year instead of ‘all of them’ [Total $464 - $700] xBulbs cost [$318-$424] yElectricity: 106*15W ~ 7,000 1/2 x9,333 kWh if off 1/3 of the time xCost varies [ cents/kWh] ~ $175-$700 yTotal cost (w/o sensors) ~ [$955 - $1,740] yHow much should we be WTP for sensors if time horizon is only one year?

PS sensors analysis zWTP [$170 - $1,700] per year (NB>0) yWe basically ‘solved for’ benefit yBut our main sensitive value was elec. Cost, so range is probably [$919 - $1,129] per yr. yNo overlap in ranges - PS always better zShould consider effects over several years yCould do a better bulb replacement model zUse more ranges - Bulb cost, labor, time yCheck sensitivity of model answer to changes yFind partial sensitivity results for each yLook at spreadsheet model yThis is fairly complicated - easier way?

Sens. Analysis for Photo Sensors zSeveral built-in options from the [Win only - mac with VPC or Intel] to check sensitivity. yOne-way (one variable at a time) yTornado and spider diagrams (all at a time) yTwo-way (two at a time)

One-Way Sensitivity Analysis zUse plugin. zMakes graph that varies a single variable from a low- to high-end range and shows output (eg NPV) as a function of variable zThe resulting graph shows how “sensitive” your answer is to changes in the one variable yShows simple trend related to one variable

Tornado Diagrams zShows results of many one-way plots on single chart (one changes while all others held constant) zLength of bars tells you how sensitive output is to each variable yBigger bar = more sensitive ySoftware typically “sorts” most to least yLooks like a tornado zSpider diagrams are similar

Plug-in notes zSee Clemen pp.193- for TopRank tutorial zSee course web page for instructions o using treeplan plugins (eg SensIt) yRead the PDFs for install instructions! yShould install ok on most cluster computers yWatch for “macro security privilege” warning. xIf you get it, change setting under tools->Macro menu (don’t necessarily need to use lowest setting)

Two-way SA zShows a 2-dimensional plot of what happens when we change 2 variables at once yGraph generated is a “frontier” of feasible options between two variables

EXCEL’s TABLE function zOne- and two-input data tables zSort of a built-in tool for sensitivity analysis (without fancy graphs/etc). ySee PDF posted for examples and instructions yTopRank type analysis typically easier, however EXCEL TABLE requires no plug in (or expensive software)