1. Uncertainty budget
In many situations we have uncertainties come from several sources.
When the total uncertainty is too large, we look for ways of reducing it.
The tolerable total uncertainty is our uncertainty budget, and we need to achieve it by reducing individual uncertainties in the most cost effective way.
This will be illustrated by a study by Chanyoung Park on deciding between reducing uncertainties at the material level or at the structural level.

2. Chanyoung Park, Raphael T. Haftka, and Nam-Ho Kim
Modeling the Effect of Structural Tests on Uncertainty in Estimated Failure Stress (Strength)
Chanyoung Park, Raphael T. Haftka, and Nam-Ho Kim
Presentation based on
Park, C., Matsumura, T., Haftka, R. T., Kim, N. H. and Acar.,E., “ Modeling the effect of structural tests on uncertainty in estimated failure stress “ 13th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Fort Worth, Texas, Sept , 2010

3. Multistage testing for design acceptance
Building-block process
Detect failures in early stage of design
Reduce uncertainty and estimate material properties
A large number of tests in lower pyramid (reducing uncertainty)
System-level probability of failure controlled in upper pyramid (certification)
ELEMENTS
DETAILS
COMPONENTS
COUPONS
SYSTEM
DATA BASE
STRUCTURAL FEATURES
GENERIC SPECIMENS
NON-GENERIC SPECIMENS
This presentation’s scope is first two stages coupon tests and element tests. The coupon tests establish the distribution of the material strength, and the element tests establish the accuracy of the failure theory used to predict element failure from material strength.
For the entire pyramid, the final goal is certifying the system by a certification test which demonstrates that the wing, for example, can carry loads that are high enough to keep the probability of failure in flight acceptably low.

4. Uncertainty in element strength estimates
Structural elements are under multi-axial stress and element strength has variability (aleatory uncertainty)
Element strength is estimated from material strengths in different directions using failure theory, which is not perfectly accurate (epistemic uncertainty)
Material coupon tests are done to characterize the aleatory uncertainty, but with finite number of tests we are left with errors in distributions (epistemic uncertainty)
Element tests reduce the uncertainty in failure theory.
If we can tolerate a certain total uncertainty we need to decide on number of coupon and element tests.
Our goal is to characterize well the distribution of element strength, typically the mean and standard deviation.
We have two main sources of error, which are both epistemic uncertainties. First, to find the distribution of material strength we perform coupon tests, and we cannot afford to have an infinite number of these, so we are left with sampling errors.
Second, we calculate the element strength from a failure theory that combines strengths in different directions, because the element has multi-axial loading, and stresses that vary from point to point.
The challenge addressed in this lecture is how to decide on the balance between the number of coupon tests and number of element tests.
ELEMENT
COUPON

5. Estimating mean and STD of material strength
Goal: Estimate distribution of material strength from nc samples
Assumption: true material strength: tc,true ~ N(mc,true, sc,true)
Sample mean & STD: (mc,test, sc,test)
Predicted mean & STD
tc,true
mc,true
mc,test
tc,test
tc,P
When we estimate the distribution of material strength from nc samples, we expect to be a bit off. The red curve depicts the true distribution of material strength, while the purple curve depicts the one based on the measured mean and standard deviation.
Fortunately, we have calculated the predictive distribution of the mean and standard deviations of a normal distribution from nc samples. The mean follows a normal distribution with the same mean but with a standard deviation which is smaller by the square root of the number of samples. The standard deviation follows a chi distribution.
tc,P ~ N(mc,P, sc,P)
Distribution of distributions!!

6. Top Hat question
In tests of 50 samples, the mean strength was 100 and the standard deviation of the strength was 7. What is the typical distance between the red and purple curves in the figure 7 1
0.7
tc,true
mc,true
mc,test
tc,test
tc,P

7. Obtaining the predictive strength distribution by sampling?
Predictive distribution of material strength
tc,P ~ N(mc,P, sc,P)
Predictive mean & STD
Samples of possible material strength distributions
Predictive true material strength distribution
How do we decide how many samples?
Sampling
mc,P
sc,P
tc,P
To obtain the predictive distribution (denoted by P) of the strength, we can use sampling. We sample a mean, then we sample a standard deviation, and then we use those to sample one strength. This is repeated until we have enough samples to get the predictive distribution of the strength.

8. Example (predictive material strength)
Predictive distribution of material strength w.r.t. # of specimens
tc,P ~ N(mc,P, sc,P)
# of samples 30 80
μc,test
1.053
1.113
σc,test
0.096
0.083
Std. μc,P
0.018
0.009
Std. σc,P
0.013
0.007
Mean τc,P
Std. τc,P
0.098
0.085
μc,true
1.1
σc,true
0.077
0.6
0.7
0.8
0.9 1
1.1
1.2
1.3
1.4
1.5
1.6
tc,P
nc = 30
nc = 80
tc,true
This is an example where the true mean is 1.1, and the standard deviation of material strength is (7% coefficient of variation). The observed mean and standard deviations with 30 samples are somewhat extreme compared to the noise that we would expect. With 30 samples, the standard deviation of the mean with repeated samples is 0.018, so that that is almost three standard deviations away. The standard deviation of the standard deviation is 0.013, so that is a couple of standard deviation away.
τc,P is biased but it compensates by wider distribution
Are any of these results extreme compared to the expected scatter?

9. Estimating element strength
Failure theory
te,true = ktruetc,true
Coupon strength
Element strength
Assumption: true element strength te,true ~ N(me,true, se,true)
Error in failure theory
Element tests used to reduce errors using Bayesian updating s2 s1
tc,true
te,true
Multiaxial loading
Failure envelope
ktrue
me,true = ktruemc,true
se,true = ?
me,P= (1 – ek)kcalcmc,P
Errors
se,P = (1 – es)sc,P
To predict element strength we need to deal with the complication that the element is under multi-axial loading so that several stress components act simultaneously. This requires us to use some kind of failure theory in order to apply material strength to element failure.
For isotropic materials, we test strength only in one direction, and we can use a failure theory, such as maximum shear stress (Tresca) to predict failure under multi-axial stresses. For anisotropic materials, we test strength in several directions and use a more complex failure theory such as Tsai-Wu.
This is shown schematically in the figure for the case of two stress components. The strength in the 1 and 2 directions have been obtained from coupon tests, and a failure theory has been used to construct a failure envelope shown in red. The element is loaded in some intermediate direction as shown by the arrow.
We assume that element strength is normally distributed, that he standard deviation is not known, but that the mean is obtained from the failure theory which produces a correction k to the material mean strength. The failure theory is not perfectly accurate, so we have a calculated k, which is different from the true k. Similarly, the standard deviation is likely to be close to that of the material strength but there is some difference that we also denote as error.

10. Prior distribution (mean element strength)
me,P = (1 – ek)kcalcmc,P
Uniform distribution for error in failure theory (±10%)
Used Bayesian network to calculate PDF of me,Ptrue
Similar calculation for element std. nc
mc,P
Before we take into account the element tests, we construct a prior distribution for the mean and standard deviation. This slides shows the process for the mean. We have the predictive distribution of the mean of the material strength mu_cP, and we have the distribution of error, which was assumed to be +-10% in the following calculations.
This two can be combined by Mote Carlo sampling to yield the prior for the mean element strength. In the paper they were combined analytically using a Bayesian network and a convolution integral.
For the standard deviation we have exactly the same process.
mc,test
sc,test
me,P ek

11. Error in Failure theory
Prior example
Prior distribution (Joint PDF) for the mean and STD of element strength
Error
Distribution
Bounds
Error in Failure theory
Uniform
±10%
Error in estimated std.
±50%
200x200 grid
me,P
se,P
mc,P
sc,P
Joint PDF
me,P
se,P
0.05
0.1
0.15
0.2
0.25
0.9
1.0
1.1
1.2
1.3
Uncertainty increases due to error (epistemic uncertainty)
This is an example of assumed error distribution in our knowledge of the mean and standard deviation of element strength. The error in the failure theory is assumed to be relatively small +-10%, but the difference between the coupon and element standard deviation may be more substantial and we assume differences up to 50%.
The mean and standard deviations are independent so the joint distribution shown in the right figure is the product of the two individual distribution. The contours are based on a dense grid where we compute the joint pdf.

12. Bayesian update me,P me,true Element test ~ N(me,true, se,true)
Update the joint PDF (me,P, se,P) using ne element tests
Reduce epistemic uncertainty in ek
fM(m | test) = L(test | m)fM(m)
The process of using element tests for Bayesian updating is illustrated here. Each test result is used to update the distribution using Bayes rule. Unlike the previous slide this one shows the effect of Bayesian updating only on the mean.
In the illustration three test results are shown by the green, light blue and dark blue circles. The resulting updated distributions are also shown. After three tests the distribution of the mean is much narrower.
Note that this slides has animations.
me,P
me,true

13. Illustration of convergence of coupon mc,P & sc,P
True distribution of material strength
Test
Distribution
Parameters
Coupon test
Normal
mc,true = 1.1, sc,true = 0.077
Estimated mean of (mc,P & sc,P) (single set cumulative)
This is another example of the variation of the mean and standard deviation from a single set, which is different from the one shown on Slide 8. It intends to show that as we increase the number of coupon tests, the error does not reduce monotonically for a single set because of the randomness of the sampling process.

14. Illustrative example (coupon tests)
Estimated STD of (mc,P & sc,P)
While the actual error does not reduce monotonically, as shown on the previous slide, the uncertainty estimates do converge well.
Increasing nc reduces uncertainties in the estimated parameters
Effectiveness of reducing uncertainty is high at low nc

15. Top Hat question
Why does the convergence on Slide 14 (uncertainty estimates) look so much better than the convergence on Slide 13 (estimates of mean and standard deviations)
Noise to signal ratio
Difference in scales
Both

16. Illustrative element updated distribution
True distribution of element strength
Test
Distribution
Parameters
Element test
Normal
me,true = 1.1, se,true = 0.099
Updated joint PDF of parameters
se,true
ML me,P
me,true
ML se,P
me,P
se,P
This is an illustration how one element test changes the distribution of the predictive mean and predictive standard deviation.

17. Element updated distributions for 10 coupon tests
RMS error (500K instances of tests) vs. uncertainty in mean and standard deviation from a single set of tests
STD of mean
STD of STD
These figures compare our estimate of the uncertainty in the mean and standard deviation of element strength from a single set of 10 coupon tests and a single set of element tests (in color) and what we get for the rms error of the estimate over 500,000 sets of tests results.

18. Element updated distributions – 90 coupon tests
RMS error (500K MCS) vs. estimated uncertainty in means and standard deviation from a single set of tests.
STD of mean
STD of STD
For 90 coupon tests, the uncertainty in the mean hardly changes with number of element tests, because for our example the mean of the coupon strength is the same as the mean of the element strength (at 1.1). The standard deviations are different (0.077 vs ), so adding element tests narrows down the uncertainty and the errors.

19. Dependence of rms errors on number of tests
Mean of element strength
STD of element strength
Figure illustrates the effect of diminishing returns. There is substantial reduction in errors going from 10 coupon tests to 50, but hardly any change going from 50 to 90. Similarly, there is substantial reduction in error with the first element test, but much less thereafter.
First element test has a substantial effect to reduce uncertainty in estimated parameters of element strength