1. 1 Managing the Computational Cost in a Monte Carlo Simulation by Considering the Value of Information E. Nikolaidis, V. Pandey, Z. P. Mourelatos April 23, 2012

2. 2 Outline 1.Definition and Significance 2.Objective 3.Approach 4.Example 5.Conclusions

3. 3 1. Definition and Significance Computers are becoming faster; so does the need for computing power One million replications to estimate a probability of 10 -3

4. 4 Solutions Faster computers Better methods/software Assess the added value of the information from a Monte Carlo simulation before performing it.

5. 5 Develop structured method to quantify expected value of information –Based on first principles –Concrete measure of value information Scope: Monte Carlo simulation for calculation of reliability 2. Objective

6. 6 Principles We perform Monte-Carlo simulation to choose the best design among alternatives Value of information depends on particular decision Value of information = Payoff with information-Payoff without it

7. 7 Scope: decisions where reliability of a design is the only important attribute Estimate reliability using Monte Carlo simulation Finite replications: Uncertainty in reliability 3. Approach Using Decision Analysis to Measure the Value of Information from a Simulation

8. 8 Problem Formulation Estimate expected utility (or Certain Equivalent reliability) vs. number of replications, before performing simulation, for the following decision: Designs A, B,…: Select the most reliable

9. 9 Steps 1.Estimate Prior Probability Density Functions (PDFs) of reliabilities (or failure probabilities) 2.Elicit U(r): measures value of a design with reliability r 3.Estimate value of decision without information, and with perfect information. 4.Determine value of incomplete information vs. replications.

10. 10 Value of decision before simulation Expected utility of design A Expected utility of design B

11. 11 Value with perfect information True failure probabilities  Choose most reliable design with perfect confidence True failure probabilities pApA Calculate expected utilities of alternative designs Select design pBpB EU(p A ) EU(p B ) EU(p A ) EU(p B ) A better than B B better than A

12. 12 Added Value of Perfect Information (continued) Added value of perfect information Expected utility with perfect information Do not pay more than EVPI for information

13. 13 Estimating Value of Information Generate sample values or failure probabilities of the two designs Generate sample values of numbers of observed failures (evidence) in simulation Update PDFs of failure probabilities Calculated expected utilities, select best design Estimate expected utility and CE of decision

14. 14 Alternative Designs A and B Know prior PDF of failure probability, and utility function of reliability How many replications should we perform? 4. Example

15. 15 Prior PDF’s of Reliabilities Design B is more reliable than A with probability 0.82. Beta(1.5, 10501) Beta(0.5,10502)

16. 16 Updated PDF’s after performing 10 4 (continuous) and 10 5 replications (dashed) 10 5 replications: P(B more reliable than A)=0.99

17. 17 Value of decision increases with the replications. Point of diminishing returns: 40,000 replications. Value of information from 40,000 replications Failure probability

18. 18 6. Conclusions Proposed and demonstrated structured method to assess value of information from Monte Carlo simulation. Application of method does not require any analysis of the system. Value of information = increase in utility resulting from the use of the information. Finding the point of diminishing returns helps manage computational cost.