Estimation and Uncertainty H. Scott Matthews / Lecture 10 - Oct. 1, 2003

Problem of Unknown Numbers  If we need a piece of data, we can:  Look it up in a reference source  Collect number through survey/investigation  Guess it ourselves  Get experts to help you guess it  Often only ‘ballpark’, ‘back of the envelope’ or ‘order of magnitude needed  Situations when actual number is unavailable or where rough estimates are good enough  E.g. 100s, 1000s, … (10 2, 10 3, etc.)  Source: Mosteller handout

Uncertainty zInvestment planning and benefit/cost analysis is fraught with uncertainties yforecasts of future are highly uncertain yapplications often made to preliminary designs ydata is often unavailable zStatistics has confidence intervals – economists need them, too.

Sensitivity Analysis zMost of our discussions and examples have been simple (e.g. y=mx+b) zLife is not as simple as this  E.g. y=  x  zWe need to be able to create models and methods that can incorporate our uncertainties

Definition: “Base Case” zGenerally uses single values and our ‘best guesses’ zSensitivity Analysis acknowledges uncertainty exists zIncorporate variables instead of constant assumptions zIf our ‘Net Benefits’ remain positive over a wide range of reasonable assumptions, then robust results

How many variables? zChoosing ‘variables’ instead of ‘constants’ for all parameters is likely to make model unsolvable zPartial sens. Analysis - change only 1  Equivalent of  y/  x yDo for the most ‘critical’ assumptions yCan use this to find ‘break-evens’

Best and Worst-Case Analysis zDoes any combination of inputs reverse the sign of our answer? yIf so, are those inputs reasonable? yE.g. using very conservative ests. yMight want NB > 0, but know when NB < 0 ySimilar to ‘breakeven analysis’

Question 2.4 from Text z3 projects being considered R, F, W yRecreational, forest preserve, wilderness yWhich should be selected?

Question 2.4 Project “R with Road” has highest NB

Question 2.4 w/ uncertainty zWhat if we note that Benefits/Costs of each project are uncertain by plus or minus 10%? ye.g. instead of Project R having benefits of $10 million, could be as low as $9 million or as high as $11 million yRepeat for all project combinations zNow which project is ‘best’?

Question 2.4 w/ uncertainty Best Case: R w/Road (same) Worst Case: Road Only But difficult to determine that from this chart - can we do better?

Using error bars zShow ‘original’ point as well as range of uncertainty associated with point yRange could be fixed number, percentage, standard deviation, other zExcel tutorial available at: yhttp://phoenix.phys.clemson.edu/tutorials/ex cel/advgraph.html ySee today’s spreadsheet on home page yGraphs original points, and min/max deviations from that as error bars…

Error bar result Easier to see ‘best case/base case/worst case’ results - imagine moving a straight edge vertically up and down the axis to see result.

Case: Photo-sensors for lighting zSave electricity by installing sensors in areas where natural light exists ySensors ‘see’ light, only turn light fixtures on when needed zMAIN Posner 2nd floor hallway uses watt fluorescent bulbs for 53 fixtures zHow could we make a model to determine whether this makes sense? yAssume only one year time frame

Photo-sensors for lighting zAssume we only care about ‘one year project’ zCosts = Labor cost, installation cost, electricity costs, etc. yAssume each bulb costs $6 zBenefits = ? How should we set up model? yAssume equal, set up as ‘show minimum cost’ option zCase 1 ‘Status quo’: assume lights used as is yOn all the time, bulbs last 10,000 hours ~ burn out once per year) zCase 2 ‘PS’: pay to install sensors now, bulbs off between 1/3 and 1/2 of time

Lighting Case Study - Status Quo zCosts(sq) - lights on all the time yLabor cost: cost of replacing used bulbs x“How many CMU facilities employees does it take to change a light bulb?” - and how long does it take? xAssume labor cost = $35/hr, 15 mins/bulb x26.5 hours to change all bulbs each year, for a total labor cost of $927.50! Also, bulb cost $636/yr yElectricity: 106*15W ~ 14,000 kWh/yr (on 24-7) xCost varies from cents/kWh ~ $350-$1045 xCost Replacing bulbs is same ‘order of magnitude’ as the electricity! (Total range [$1,911 - $2,608])

Lighting Case Study - PS sensors zCosts(ps) - probably ‘off’ 1/3 - 1/2 of time yLabor cost: cost of installing sensors = ‘unknown’ yLabor cost: cost of installing new bulbs xCould assume 1/2 - 2/3 of bulbs changed per year instead of ‘all of them’ [Total $464 - $700] xBulbs cost [$318-$424] yElectricity: 106*15W ~ 7,000 1/2 x9,333 kWh if off 1/3 of the time xCost varies [ cents/kWh] ~ $175-$700 yTotal cost (w/o sensors) ~ [$955 - $1,740] yHow much should we be WTP for sensors if time horizon is only one year?

PS sensors analysis zWTP [$170 - $1,700] per year (NB>0) yWe basically ‘solved for’ benefit yBut our main sensitive value was elec. Cost, so range is probably [$919 - $1,129] per yr. yNo overlap in ranges - PS always better zShould consider effects over several years yCould do a better bulb replacement model zUse more ranges - Bulb cost, labor, time yCheck sensitivity of model answer to changes yFind partial sensitivity results for each yLook at spreadsheet model yThis is fairly complicated - easier way?

Monte Carlo Sens. Anal. zMonte Carlo analysis’ 3 steps yFirst, specify probability distributions ySecond, trial by random draws (plug them in) yThird, repeat for many (000s) of trials xProduces some distribution of results yLaw of large numbers says convergence ySee Appendix 7A for spreadsheet tutorial yhttp:// xSee sens anal and risk sim excel add-ins, PDFs

Final Notes on Uncertainty zIt is inherent to everything we do zOur goal then is to best understand and model its existence, make better results zWe ‘internalize’ the uncertainty by making ranges or distributions of variables zWe see the effects by performing sensitivity analysis (one of three methods)