1. Probabilistic Approach to Design under Uncertainty Dr. Wei Chen Associate Professor Integrated DEsign Automation Laboratory (IDEAL) Department of Mechanical Engineering Northwestern University

2. Outline Uncertainty in model-based design What is probability theory? How does one represent uncertainty? What is the inference mechanism? Connection between probability theory and utility theory Dealing with various sources of uncertainty in model-based design Summary

3. Model (lack of knowledge) Parametric (lack of knowledge, variability) Numerical Testing data Types of Uncertainty in Model-Based Design Problem faced in design under uncertainty To choose from among one set of possible design options X, where each involves a range of uncertain outcomes Y To avoid making an “illogical choice”

4. Probability theory is the mathematical study of probability. Probability derives from fundamental concepts of set theory and measurement theory. Basic Concepts of Probability Theory Event –subset of a sample space e.g., A {e 2 and e 3 } –experiments result in two different faces Probability P(e 1 )=P(e 2 )=P(e 3 )=P(e 4 )=0.25 P(A)= P(e 2 )+P(e 3 )=0.5 P(null) = 0 P(  )=1 Sample Space  Event A e3e3 e2e2 e1e1 e4e4 Example: Flip two coins Sample space  – set of all possible outcomes of a random experiment under uncertainty Outcomes {e1=HH, e2=HT, e3=TH, e4=TT}

5. Three axioms of probability measure –0  P(A)  1; P(  )=1; P(  A i )=  P(A i ) A i are disjoint events Arithmetic of probabilities –Union, Intersection, and Conditional probabilities Random variable is a function that assigns a real number to each outcome in the sample space Probability density function & arithmetic of moments of a random variable, e.g., E[XY]=E[X]E[Y] if X and Y are independent Convergence (law of large numbers) and central limit theorem Mathematics in Probability Theory Example: define x = total number of heads among the two tosses Possible values {X=0}={TT}; {X=1}={HT, TH}, {X=2}={HH} P{X=1}=0.5

6. Probabilistic Design Metrics in Quality Engineering Robustness 0 Probability Density (pdf) Performance y Target M Bias Minimizing the effect of variations without eliminating the causes  y yy yy Performance g R=Area = Prob{g(x)  c} Reliability pdf To assure proper levels of “safety” for the system designed C

7. Frequentist –Assign probabilities only to events that are random based on outcomes of actual or theoretical experiments –Suitable for problems with well-defined random experiments Bayesian –Assign probabilities to propositions that are uncertain according to subjective or logically justifiable degrees of belief in their truth Example of proposition: “there was life on Mars a billion years ago” –More suitable for design problems: events in the future, not in the past; all design models are predictive. –More popular among decision theorists Philosophies of Estimating Probability

8. In the absence of data (experiments), we have to guess –A probability guess relies on our experience with “related” events Once data is collected, inference relies on Bayes theorem –Probabilities are always personal degrees of belief –Probabilities are always conditional on the information currently available –Probabilities are always subjective “Uncertainty of probability” is not meaningful. Bayesian Inference Bernardo, J.M. and Smith, A. F., Bayesian Theory, John Wiley, New York, 2000.

9. Bayes’ Theorem H - Hypothesis D - Data Bayes’ theorem provides A solution to the problem of how to learn from data A form of uncertainty accounting A subjective view of probability P (D | H) = L(H) Max. Likelihood. Est. Data H Belief about H before obtaining data, prior P P (H) H Prior mean Belief about H after obtaining data, posterior P P (H | D) H Posterior mean Updated by data

10. Formalism of Bayesian Statistics Offers a rationalist theory of personalistic beliefs in contexts of uncertainty with axioms clearly stated Establishes that expected utility maximization provides the basis for rational decision making Not descriptive, i.e., not to model actual behavior. Prescriptive, i.e., how one should act to avoid undesirable behavioural inconsistency

11. Three basic elements of decision –the alternatives (options) X –the predicted outcomes (performance) Y –decision maker’s preference over the outcomes, expressed as an objective function f in optimization Utility theory –Utility is a preference function built on the axiomatic basis originally developed by von Neumann and Morgenstern (1947) –Six axioms (Luce and Raiffa, 1957; Thurston, 2006)  Completeness of complete order  Transitivity  Monotonicity  Probabilities exist and can be quantified  Monotonicity of Probability  Substitution-independence Connection of Probability Theory and Utility Theory In agreement to employing probability to model uncertainty

12. Without uncertainty –objective function f = V(Y) = V(Y(X)) V - value function, e.g. profit With uncertainty –objective function f = E(U) - expected utility. The preferred choice is the alternative (lottery) that has the higher expected utility. Decision Making – Ranking Design Alternatives pdf (V) V (e.g. profit) A B U (V) V 1 0 worst best Risk neutral Risk averse Risk prone

13. Issues in Model-Based Design Chen, W., Xiong, Y., Tsui, K-L., and Wang, S., “Some Metrics and a Bayesian Procedure for Validating Predictive Models in Engineering Design”, DETC2006-99599, ASME Design Technical Conference. How should we provide probabilistic quantification of uncertainty associated with a model? How should we deal with model uncertainty (reducible) and parameter uncertainty (irreducible) simultaneously? How should we make a design decision with good confidence?

14. Bayesian Approach for Quantifying the Uncertainty of Predictive Model - Physical observation - Computer model output - Bias function (between and ) - True but unknown real performance - Random error in physical experiment Computer experiments Physical experiments Observations (data) of Metamodel of Uncertainty is accounted for by Bias-Correction and UQ Bayesian Approach and UQ Bayesian posterior of

15. mean: zero variance: Model assumption Gaussian process - - Gaussian process (I.I.D.) mean: covariance:. More about the Bayesian Approach Known parameters (deterministic) Priors distribution of parameters (nondeterministic) Estimated from data, by MLE or Cross validation or Physical experiment Data Computer experiment Posterior distribution of parameters Posterior distribution of That is, the posterior of is a non-central t process (omitted here)

16. Predictive model and uncertainty quantification Design validation metrics Design objective function and uncertainty quantification No Given computer model Yes Parameter uncertainty Specified confidence level P th Design decision Expected Utility Optimization Design validity requirements satisfied ( M D < P th )? Integrated Framework for Handling Model and Parameter Uncertainties Computer experiments Physical experiments Sequential experiment design

17. 95% PI Realizations of A robust design objective (smaller-is-better) is used to determine the optimal solutions. Uncertainty of w 1, w 2 : weighting factors Uncertainty of Apley et al. (2005) developed analytical formulations to approximately quantify the mean and variance of. In this example, Monte Carlo Simulation is employed. x is a design variable and a noise variable Uncertainty Quantification of Design Objective Function with Parameter Uncertainty Mean of

18. Three types of design validation metrics (M D ) – f is small-the-better  Type 2: Additive Metric  Type 3: Worst-Case Metric Validation Metrics  Type 1: Multiplicative Metric averaging Probabilistic measure of whether a candidate optimal design is better than other design choices with respect to a particular design objective Larger confidence Smaller confidence M D is intended to quantify the confidence of choosing x* as the optimal design among all design candidates or within design region. f x* x f x x1x1 x2x2 x1x1 x2x2

19. Summary Prediction is the basis for all decision making, including engineering design. Probability is a belief (subjective), while observed frequencies are used as evidence to update the belief. Probability theory and the Bayes theorem provide a rigorous and philosophically sound framework for decision making. Predictive models in design should be described as stochastic models. The impact of model uncertainty and parameter uncertainty can be treated separately in the process of improving the predictive capability. Probabilistic approach offers computational advantages and mathematical flexibility.