1 Managing the Computational Cost in a Monte Carlo Simulation by Considering the Value of Information E. Nikolaidis, V. Pandey, Z. P. Mourelatos April.

Slides:

Advertisements

Similar presentations

1 Uncertainty in rainfall-runoff simulations An introduction and review of different techniques M. Shafii, Dept. Of Hydrology, Feb

Advertisements

Structural reliability analysis with probability- boxes Hao Zhang School of Civil Engineering, University of Sydney, NSW 2006, Australia Michael Beer Institute.

Ordinal Optimization. Copyright by Yu-Chi Ho2 Some Additional Issues F Design dependent estimation noises vs. observation noise F Goal Softening to increase.

DETC06: Uncertainty Workshop; Evidence & Possibility Theories Evidence and Possibility Theories in Engineering Design Zissimos P. Mourelatos Mechanical.

Decision Making Under Risk Continued: Bayes’Theorem and Posterior Probabilities MGS Chapter 8 Slides 8c.

SE503 Advanced Project Management Dr. Ahmed Sameh, Ph.D. Professor, CS & IS Project Uncertainty Management.

Section 7: Confidence Bounds on Component Reliability COMPONENT RELIABILITY CONFIDENCE BOUNDS Confidence bounds on component reliability are calculated.

Simulation with ArenaAppendix C – A Refresher on Probability and StatisticsSlide 1 of 33 A Refresher on Probability and Statistics Appendix C.

Sampling Distributions (§ )

Probabilistic Re-Analysis Using Monte Carlo Simulation

Reliability Based Design Optimization. Outline RBDO problem definition Reliability Calculation Transformation from X-space to u-space RBDO Formulations.

Approaches to Data Acquisition The LCA depends upon data acquisition Qualitative vs. Quantitative –While some quantitative analysis is appropriate, inappropriate.

GoldSim 2006 User Conference Slide 1 Vancouver, B.C. The Submodel Element.

©GoldSim Technology Group LLC., 2004 Probabilistic Simulation “Uncertainty is a sign of humility, and humility is just the ability or the willingness to.

Autocorrelation and Linkage Cause Bias in Evaluation of Relational Learners David Jensen and Jennifer Neville.

Importance Sampling. What is Importance Sampling ? A simulation technique Used when we are interested in rare events Examples: Bit Error Rate on a channel,

Understanding the Accuracy of Assembly Variation Analysis Methods ADCATS 2000 Robert Cvetko June 2000.

Sensitivity Analysis for Complex Models Jeremy Oakley & Anthony O’Hagan University of Sheffield, UK.

Operations Management Decision-Making Tools Module A

USE OF LAPLACE APPROXIMATIONS TO SIGNIFICANTLY IMPROVE THE EFFICIENCY

Basics of Statistical Estimation. Learning Probabilities: Classical Approach Simplest case: Flipping a thumbtack tails heads True probability  is unknown.

Results 2 (cont’d) c) Long term observational data on the duration of effective response Observational data on n=50 has EVSI = £867 d) Collect data on.

Maximum Likelihood We have studied the OLS estimator. It only applies under certain assumptions In particular,  ~ N(0, 2 ) But what if the sampling distribution.

Probabilistic Robotics Introduction Probabilities Bayes rule Bayes filters.

A Two Level Monte Carlo Approach To Calculating

1 Transforming the efficiency of Partial EVSI computation Alan Brennan Health Economics and Decision Science (HEDS) Samer Kharroubi Centre for Bayesian.

Non-parametric Bayesian value of information analysis Aim: To inform the efficient allocation of research resources Objectives: To use all the available.

Using ranking and DCE data to value health states on the QALY scale using conventional and Bayesian methods Theresa Cain.

Value of Information for Complex Economic Models Jeremy Oakley Department of Probability and Statistics, University of Sheffield. Paper available from.

Nonlinear Stochastic Programming by the Monte-Carlo method Lecture 4 Leonidas Sakalauskas Institute of Mathematics and Informatics Vilnius, Lithuania EURO.

BA 427 – Assurance and Attestation Services

1 Efficient Re-Analysis Methodology for Probabilistic Vibration of Large-Scale Structures Efstratios Nikolaidis, Zissimos Mourelatos April 14, 2008.

1 Assessment of Imprecise Reliability Using Efficient Probabilistic Reanalysis Farizal Efstratios Nikolaidis SAE 2007 World Congress.

Reliability Model for Compressor Failure SMRE Term Project Paul Zamjohn August 2008.

Estimation and Hypothesis Testing. The Investment Decision What would you like to know? What will be the return on my investment? Not possible PDF for.

Mote Carlo Method for Uncertainty The objective is to introduce a simple (almost trivial) example so that you can Perform.

Lecture 12 Statistical Inference (Estimation) Point and Interval estimation By Aziza Munir.

1 Institute of Engineering Mechanics Leopold-Franzens University Innsbruck, Austria, EU H.J. Pradlwarter and G.I. Schuëller Confidence.

Probabilistic Mechanism Analysis. Outline Uncertainty in mechanisms Why consider uncertainty Basics of uncertainty Probabilistic mechanism analysis Examples.

An Empirical Likelihood Ratio Based Goodness-of-Fit Test for Two-parameter Weibull Distributions Presented by: Ms. Ratchadaporn Meksena Student ID:

1 6. Reliability computations Objectives Learn how to compute reliability of a component given the probability distributions on the stress,S, and the strength,

West Virginia University Towards Practical Software Reliability Assessment for IV&V Projects B. Cukic, E. Gunel, H. Singh, V. Cortellessa Department of.

1 SMU EMIS 7364 NTU TO-570-N Inferences About Process Quality Updated: 2/3/04 Statistical Quality Control Dr. Jerrell T. Stracener, SAE Fellow.

Computer Science, Software Engineering & Robotics Workshop, FGCU, April 27-28, 2012 Fault Prediction with Particle Filters by David Hatfield mentors: Dr.

Estimates of Global Sea Level Rise from Tide Gauges Sea level trend,

AMERICA’S ARMY: THE STRENGTH OF THE NATION Mort Anvari 1 Cost Risk and Uncertainty Analysis MORS Special Meeting | September.

Week 71 Hypothesis Testing Suppose that we want to assess the evidence in the observed data, concerning the hypothesis. There are two approaches to assessing.

Uncertainty Management in Rule-based Expert Systems

Uncertainty in Expert Systems

Statistical Decision Theory Bayes’ theorem: For discrete events For probability density functions.

Artificial Intelligence CIS 342 The College of Saint Rose David Goldschmidt, Ph.D.

Bayesian Statistics and Decision Analysis

Machine Design Under Uncertainty. Outline Uncertainty in mechanical components Why consider uncertainty Basics of uncertainty Uncertainty analysis for.

Point Pattern Analysis

Introduction to hypothesis testing Hypothesis testing is about making decisions Is a hypothesis true or false? Ex. Are women paid less, on average, than.

1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour.

1 Chapter 8: Model Inference and Averaging Presented by Hui Fang.

Optimization formulation Optimization methods help us find solutions to problems where we seek to find the best of something. This lecture is about how.

Stat 31, Section 1, Last Time Distribution of Sample Means –Expected Value  same –Variance  less, Law of Averages, I –Dist’n  Normal, Law of Averages,

Descriptive and Inferential Statistics Descriptive statistics The science of describing distributions of samples or populations Inferential statistics.

March 7, Using Pattern Recognition Techniques to Derive a Formal Analysis of Why Heuristic Functions Work B. John Oommen A Joint Work with Luis.

Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 7 Inferences Concerning Means.

Statistical Concepts Basic Principles An Overview of Today’s Class What: Inductive inference on characterizing a population Why : How will doing this allow.

Canadian Bioinformatics Workshops

Statistical Inference

Null Hypothesis Testing

More about Posterior Distributions

Hypothesis Testing.

STA 291 Spring 2008 Lecture 18 Dustin Lueker.

Maximum Likelihood We have studied the OLS estimator. It only applies under certain assumptions In particular,  ~ N(0, 2 ) But what if the sampling distribution.

Presentation transcript:

1 Managing the Computational Cost in a Monte Carlo Simulation by Considering the Value of Information E. Nikolaidis, V. Pandey, Z. P. Mourelatos April 23, 2012

2 Outline 1.Definition and Significance 2.Objective 3.Approach 4.Example 5.Conclusions

3 1. Definition and Significance Computers are becoming faster; so does the need for computing power One million replications to estimate a probability of 10 -3

4 Solutions Faster computers Better methods/software Assess the added value of the information from a Monte Carlo simulation before performing it.

5 Develop structured method to quantify expected value of information –Based on first principles –Concrete measure of value information Scope: Monte Carlo simulation for calculation of reliability 2. Objective

6 Principles We perform Monte-Carlo simulation to choose the best design among alternatives Value of information depends on particular decision Value of information = Payoff with information-Payoff without it

7 Scope: decisions where reliability of a design is the only important attribute Estimate reliability using Monte Carlo simulation Finite replications: Uncertainty in reliability 3. Approach Using Decision Analysis to Measure the Value of Information from a Simulation

8 Problem Formulation Estimate expected utility (or Certain Equivalent reliability) vs. number of replications, before performing simulation, for the following decision: Designs A, B,…: Select the most reliable

9 Steps 1.Estimate Prior Probability Density Functions (PDFs) of reliabilities (or failure probabilities) 2.Elicit U(r): measures value of a design with reliability r 3.Estimate value of decision without information, and with perfect information. 4.Determine value of incomplete information vs. replications.

10 Value of decision before simulation Expected utility of design A Expected utility of design B

11 Value with perfect information True failure probabilities  Choose most reliable design with perfect confidence True failure probabilities pApA Calculate expected utilities of alternative designs Select design pBpB EU(p A ) EU(p B ) EU(p A ) EU(p B ) A better than B B better than A

12 Added Value of Perfect Information (continued) Added value of perfect information Expected utility with perfect information Do not pay more than EVPI for information

13 Estimating Value of Information Generate sample values or failure probabilities of the two designs Generate sample values of numbers of observed failures (evidence) in simulation Update PDFs of failure probabilities Calculated expected utilities, select best design Estimate expected utility and CE of decision

14 Alternative Designs A and B Know prior PDF of failure probability, and utility function of reliability How many replications should we perform? 4. Example

15 Prior PDF’s of Reliabilities Design B is more reliable than A with probability Beta(1.5, 10501) Beta(0.5,10502)

16 Updated PDF’s after performing 10 4 (continuous) and 10 5 replications (dashed) 10 5 replications: P(B more reliable than A)=0.99

17 Value of decision increases with the replications. Point of diminishing returns: 40,000 replications. Value of information from 40,000 replications Failure probability

18 6. Conclusions Proposed and demonstrated structured method to assess value of information from Monte Carlo simulation. Application of method does not require any analysis of the system. Value of information = increase in utility resulting from the use of the information. Finding the point of diminishing returns helps manage computational cost.