1. Software Tools for Sensitivity and Uncertainty Analysis H. Scott Matthews 12-706/73-359 Lecture 11 - Oct. 6, 2004

2. Admin Issues zReminder: Discounting/Finance Review on Friday - HERE 10:30am zHW 2 back on Monday zProject feedback coming

3. Final Notes on Uncertainty zIt is inherent to everything we do zOur goal then is to best understand and model its existence, make better results zWe ‘internalize’ the uncertainty by making ranges or distributions of variables zWe see the effects by performing sensitivity analysis (one of three methods)

4. From Last Time: Error bar result Easier to see ‘best case/base case/worst case’ results - imagine moving a straight edge vertically up and down the axis to see result.

5. Reconsider Photo Sensor Example z(Still using Sens Anal XLS file on web) zLooked last time at ‘Photo’ worksheet yAs we add scenarios, the model/spreadsheet becomes increasingly complicated yAdding new columns for different inputs zHowever, still a straightforward method zWe can easily replicate numbers from lecture in this excel file

6. Adding scenarios/sensitivity z“What if” we care about how the total costs of the base case vary as the price of electricity changes? z“What if” we care about how the percent time natural light is available (i.e. percent time sensors on/off) affects cost of PS? zCan do this quickly and easily with tools zUse ‘Sensit’ excel add-in from last time yhttp://www.treeplan.com/http://www.treeplan.com/ yDownload.xla (add-in),.xls (example),.pdf (guide)

7. Sensit Add-in Installation Tips zSee course web page for download link zChoose Install option 3 in PDF - “put in Excel Startup folder” zClick “I agree” when sensit dialog comes up on startup zSee ‘sens analysis’ spreadsheet from last class for example outputs

8. Types of Results in Sensit zPlot : partial sensitivity analysis with one variable zSpider: aggregate result of several partial analyses run separately - all displayed on same figure zTornado: aggregate result of several partial analyses run separately - all displayed on same figure zUsage: set up spreadsheet framework and formulas, choose “Tools -> Sensitivity Analysis” menu items above

9. Sensitivity Analysis Plots zUseful for single, what-if questions zTypically “partial” - effect of change in output from 1 input yCalculates/graphs results zWe see same ‘results’ as before, but can use a simpler excel model (less columns) yCompare “Photo” worksheet vs. “Photo-Sens Anal” worksheet

10. Spider/Tornado Diagrams zShow changes in output from all selected inputs by percentage, unit changes, etc yDoes sensitivity analysis ‘over a range’ for each input you specify yShows ‘output at each point in the range’ yCreates useful visualization yAids in identifying ‘key parameters’ zJust need to be careful in selecting equivalent input ranges so comparison is fair

11. Notes on Photo Sensor Case zNot so analytically interesting because all inputs have positive contribution to cost yi.e. linear model is X + Y +Z, not X -Y +Z zSee sample file that comes with add-in ySimple sales/production cost model zSpider and tornado diagrams more useful

12. Monte Carlo Sens. Anal. zMonte Carlo analysis’ 3 steps yFirst, specify probability distributions ySecond, trial by random draws (plug them in) yThird, repeat for many (000s) of trials xProduces some distribution of results yDoing Monte Carlo doesn’t “give you the answer”! xLaw of large numbers says convergence ySee Appendix 7A for spreadsheet tutorial

13. Monte Carlo Simulations zDownload RiskSim add-in from website ySee special warnings/notes in BOLD!! yIf Excel error msg, choose “no” on update links yMay need admin privileges to install zAdds special probability functions to excel yExcel has some, but these are better yAlso adds monte carlo math to excel zDistributions: Binomial, Cumulative, Discrete, Exponential, Integer, Normal, Poisson, Triangular, Uniform yCan do almost anything with these as base

14. Using Monte Carlo Methods zInstead of our ‘point estimates’ from last time, we consider probabilistic functions yPhoto-RiskSim worksheet as starting point yBulb and elec. cost obvious to do with prob. functions yOther variables are constant (e.g., number of bulbs) yAssume bulb cost is triangular, 3.5, 6, 10 yElec cost normal with mean 0.05, stdev.005 yNow what kind of result will we get? xUse with “Tools-> Risk Simulation” (one output) xNote demo version limited to 300 Monte Carlo trials yAgain, photo sensor case not ideal for this - look at production cost example

15. Wrap-Up zLook at effect of number of trials (50 - 300) yRiskSim Summary worksheet shows 300 trials - note if you do it again, results would be different! zCould do all of this with excel, would just be harder (as usual) zAdd-ins like this (or CrystalBall, @RISK) simplify this kind of analysis - you should definitely use these instead. zWe have much better models - and knowledge of our results now