Uncertainty budget In many situations we have uncertainties come from several sources. When the total uncertainty is too large, we look for ways of reducing it. The tolerable total uncertainty is our uncertainty budget, and we need to achieve it by reducing individual uncertainties in the most cost effective way. This will be illustrated by a study by Chanyoung Park on deciding between reducing uncertainties at the material level or at the structural level.

Chanyoung Park, Raphael T. Haftka, and Nam-Ho Kim Modeling the Effect of Structural Tests on Uncertainty in Estimated Failure Stress (Strength) Chanyoung Park, Raphael T. Haftka, and Nam-Ho Kim Presentation based on Park, C., Matsumura, T., Haftka, R. T., Kim, N. H. and Acar.,E., “ Modeling the effect of structural tests on uncertainty in estimated failure stress “ 13th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Fort Worth, Texas, Sept. 13-15, 2010

Multistage testing for design acceptance Building-block process Detect failures in early stage of design Reduce uncertainty and estimate material properties A large number of tests in lower pyramid (reducing uncertainty) System-level probability of failure controlled in upper pyramid (certification) ELEMENTS DETAILS COMPONENTS COUPONS SYSTEM DATA BASE STRUCTURAL FEATURES GENERIC SPECIMENS NON-GENERIC SPECIMENS This presentation’s scope is first two stages coupon tests and element tests. The coupon tests establish the distribution of the material strength, and the element tests establish the accuracy of the failure theory used to predict element failure from material strength. For the entire pyramid, the final goal is certifying the system by a certification test which demonstrates that the wing, for example, can carry loads that are high enough to keep the probability of failure in flight acceptably low.

Uncertainty in element strength estimates Structural elements are under multi-axial stress and element strength has variability (aleatory uncertainty) Element strength is estimated from material strengths in different directions using failure theory, which is not perfectly accurate (epistemic uncertainty) Material coupon tests are done to characterize the aleatory uncertainty, but with finite number of tests we are left with errors in distributions (epistemic uncertainty) Element tests reduce the uncertainty in failure theory. If we can tolerate a certain total uncertainty we need to decide on number of coupon and element tests. Our goal is to characterize well the distribution of element strength, typically the mean and standard deviation. We have two main sources of error, which are both epistemic uncertainties. First, to find the distribution of material strength we perform coupon tests, and we cannot afford to have an infinite number of these, so we are left with sampling errors. Second, we calculate the element strength from a failure theory that combines strengths in different directions, because the element has multi-axial loading, and stresses that vary from point to point. The challenge addressed in this lecture is how to decide on the balance between the number of coupon tests and number of element tests. ELEMENT COUPON

Estimating mean and STD of material strength Goal: Estimate distribution of material strength from nc samples Assumption: true material strength: tc,true ~ N(mc,true, sc,true) Sample mean & STD: (mc,test, sc,test) Predicted mean & STD tc,true mc,true mc,test tc,test tc,P When we estimate the distribution of material strength from nc samples, we expect to be a bit off. The red curve depicts the true distribution of material strength, while the purple curve depicts the one based on the measured mean and standard deviation. Fortunately, we have calculated the predictive distribution of the mean and standard deviations of a normal distribution from nc samples. The mean follows a normal distribution with the same mean but with a standard deviation which is smaller by the square root of the number of samples. The standard deviation follows a chi distribution. tc,P ~ N(mc,P, sc,P) Distribution of distributions!!

Top Hat question In tests of 50 samples, the mean strength was 100 and the standard deviation of the strength was 7. What is the typical distance between the red and purple curves in the figure 7 1 0.7 tc,true mc,true mc,test tc,test tc,P

Obtaining the predictive strength distribution by sampling? Predictive distribution of material strength tc,P ~ N(mc,P, sc,P) Predictive mean & STD Samples of possible material strength distributions Predictive true material strength distribution How do we decide how many samples? Sampling mc,P sc,P tc,P To obtain the predictive distribution (denoted by P) of the strength, we can use sampling. We sample a mean, then we sample a standard deviation, and then we use those to sample one strength. This is repeated until we have enough samples to get the predictive distribution of the strength.

Example (predictive material strength) Predictive distribution of material strength w.r.t. # of specimens tc,P ~ N(mc,P, sc,P) # of samples 30 80 μc,test 1.053 1.113 σc,test 0.096 0.083 Std. μc,P 0.018 0.009 Std. σc,P 0.013 0.007 Mean τc,P Std. τc,P 0.098 0.085 μc,true 1.1 σc,true 0.077 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 tc,P nc = 30 nc = 80 tc,true This is an example where the true mean is 1.1, and the standard deviation of material strength is 0.077 (7% coefficient of variation). The observed mean and standard deviations with 30 samples are somewhat extreme compared to the noise that we would expect. With 30 samples, the standard deviation of the mean with repeated samples is 0.018, so that that 1.053 is almost three standard deviations away. The standard deviation of the standard deviation is 0.013, so that 0.096 is a couple of standard deviation away. τc,P is biased but it compensates by wider distribution Are any of these results extreme compared to the expected scatter?

Estimating element strength Failure theory te,true = ktruetc,true Coupon strength Element strength Assumption: true element strength te,true ~ N(me,true, se,true) Error in failure theory Element tests used to reduce errors using Bayesian updating s2 s1 tc,true te,true Multiaxial loading Failure envelope ktrue me,true = ktruemc,true se,true = ? me,P= (1 – ek)kcalcmc,P Errors se,P = (1 – es)sc,P To predict element strength we need to deal with the complication that the element is under multi-axial loading so that several stress components act simultaneously. This requires us to use some kind of failure theory in order to apply material strength to element failure. For isotropic materials, we test strength only in one direction, and we can use a failure theory, such as maximum shear stress (Tresca) to predict failure under multi-axial stresses. For anisotropic materials, we test strength in several directions and use a more complex failure theory such as Tsai-Wu. This is shown schematically in the figure for the case of two stress components. The strength in the 1 and 2 directions have been obtained from coupon tests, and a failure theory has been used to construct a failure envelope shown in red. The element is loaded in some intermediate direction as shown by the arrow. We assume that element strength is normally distributed, that he standard deviation is not known, but that the mean is obtained from the failure theory which produces a correction k to the material mean strength. The failure theory is not perfectly accurate, so we have a calculated k, which is different from the true k. Similarly, the standard deviation is likely to be close to that of the material strength but there is some difference that we also denote as error.

Prior distribution (mean element strength) me,P = (1 – ek)kcalcmc,P Uniform distribution for error in failure theory (±10%) Used Bayesian network to calculate PDF of me,Ptrue Similar calculation for element std. nc mc,P Before we take into account the element tests, we construct a prior distribution for the mean and standard deviation. This slides shows the process for the mean. We have the predictive distribution of the mean of the material strength mu_cP, and we have the distribution of error, which was assumed to be +-10% in the following calculations. This two can be combined by Mote Carlo sampling to yield the prior for the mean element strength. In the paper they were combined analytically using a Bayesian network and a convolution integral. For the standard deviation we have exactly the same process. mc,test sc,test me,P ek

Error in Failure theory Prior example Prior distribution (Joint PDF) for the mean and STD of element strength Error Distribution Bounds Error in Failure theory Uniform ±10% Error in estimated std. ±50% 200x200 grid me,P se,P mc,P sc,P Joint PDF me,P se,P 0.05 0.1 0.15 0.2 0.25 0.9 1.0 1.1 1.2 1.3 Uncertainty increases due to error (epistemic uncertainty) This is an example of assumed error distribution in our knowledge of the mean and standard deviation of element strength. The error in the failure theory is assumed to be relatively small +-10%, but the difference between the coupon and element standard deviation may be more substantial and we assume differences up to 50%. The mean and standard deviations are independent so the joint distribution shown in the right figure is the product of the two individual distribution. The contours are based on a dense grid where we compute the joint pdf.

Bayesian update me,P me,true Element test ~ N(me,true, se,true) Update the joint PDF (me,P, se,P) using ne element tests Reduce epistemic uncertainty in ek fM(m | test) = L(test | m)fM(m) The process of using element tests for Bayesian updating is illustrated here. Each test result is used to update the distribution using Bayes rule. Unlike the previous slide this one shows the effect of Bayesian updating only on the mean. In the illustration three test results are shown by the green, light blue and dark blue circles. The resulting updated distributions are also shown. After three tests the distribution of the mean is much narrower. Note that this slides has animations. me,P me,true

Illustration of convergence of coupon mc,P & sc,P True distribution of material strength Test Distribution Parameters Coupon test Normal mc,true = 1.1, sc,true = 0.077 Estimated mean of (mc,P & sc,P) (single set cumulative) This is another example of the variation of the mean and standard deviation from a single set, which is different from the one shown on Slide 8. It intends to show that as we increase the number of coupon tests, the error does not reduce monotonically for a single set because of the randomness of the sampling process.

Illustrative example (coupon tests) Estimated STD of (mc,P & sc,P) While the actual error does not reduce monotonically, as shown on the previous slide, the uncertainty estimates do converge well. Increasing nc reduces uncertainties in the estimated parameters Effectiveness of reducing uncertainty is high at low nc

Top Hat question Why does the convergence on Slide 14 (uncertainty estimates) look so much better than the convergence on Slide 13 (estimates of mean and standard deviations) Noise to signal ratio Difference in scales Both

Illustrative element updated distribution True distribution of element strength Test Distribution Parameters Element test Normal me,true = 1.1, se,true = 0.099 Updated joint PDF of parameters se,true ML me,P me,true ML se,P me,P se,P This is an illustration how one element test changes the distribution of the predictive mean and predictive standard deviation.

Element updated distributions for 10 coupon tests RMS error (500K instances of tests) vs. uncertainty in mean and standard deviation from a single set of tests STD of mean STD of STD These figures compare our estimate of the uncertainty in the mean and standard deviation of element strength from a single set of 10 coupon tests and a single set of element tests (in color) and what we get for the rms error of the estimate over 500,000 sets of tests results.

Element updated distributions – 90 coupon tests RMS error (500K MCS) vs. estimated uncertainty in means and standard deviation from a single set of tests. STD of mean STD of STD For 90 coupon tests, the uncertainty in the mean hardly changes with number of element tests, because for our example the mean of the coupon strength is the same as the mean of the element strength (at 1.1). The standard deviations are different (0.077 vs. 0.099), so adding element tests narrows down the uncertainty and the errors.

Dependence of rms errors on number of tests Mean of element strength STD of element strength Figure illustrates the effect of diminishing returns. There is substantial reduction in errors going from 10 coupon tests to 50, but hardly any change going from 50 to 90. Similarly, there is substantial reduction in error with the first element test, but much less thereafter. First element test has a substantial effect to reduce uncertainty in estimated parameters of element strength