Chanyoung Park Raphael T. Haftka Paper Helicopter Project

Structural & Multidisciplinary Optimization Group 2/25 Problem1: Conservative estimate of the fall time  Estimating the 5 th percentile of the fall time of one helicopter  Estimating the 5 th percentile to compensate the variability in the fall time (aleatory uncertainty)  The sampling error (epistemic uncertainty)  Estimating the sampling uncertainty in the mean and the STD  Obtaining a distribution of the 5 th percentile  Taking the 5 th percentile of the 5 th percentile distribution to compensate the sampling error Sampling  t,P  t,P tPtP t 0.05,P

Structural & Multidisciplinary Optimization Group 3/25  Estimating the 5 th percentile of the fall time of first helicopter (mean 3.78, std 0.37)  100,000 5 th percentiles of fall time  Helicopter 1 of the dataset 3  Height: in  2.88 (sec) is the 5 th percentile of the histogram (a conservative estimate of the 5 th percentile of the fall time for 95% confidence) Problem1: Conservative estimate of the fall time

Structural & Multidisciplinary Optimization Group 4/25 Problem2: Predicted variability using prior  Calculating predicted variability in the fall time  We assume that the variability in the fall time is caused by the variability in the CD  The variability in the fall time is predicted using the computational model (quadratic model) and the distribution of the CD  The prior distribution represents our initial guess for the distribution of the CD Height at time t Steady state speed where

Structural & Multidisciplinary Optimization Group 5/25 Problem2: Comparing predicted variability and observed variability using prior  Area metric with the prior  Data set CD from the fall time data Helicopter1Helicopter2Helicopter3 Sample mean of CDs Sample STD of CDs Prior Mean of CD = 1 / STD of CD = 0.28

Structural & Multidisciplinary Optimization Group 6/25 Problem3: Calibration: Posterior distribution of mean and standard deviation  Estimating parameters of the CD distribution  We assume that CD of each helicopter follows the normal distribution  The parameters, CD and σ test are estimated using 10 data  Posterior distribution is obtained based on 10 fall time data  Non informative distribution is used for the standard deviation After 1 updateAfter 5 updatesAfter 10 updates

Structural & Multidisciplinary Optimization Group 7/25 Problem3: Comparing predicted variability using posterior and observed variability  Comparing MLE and sampling statistics  MCMC sampling  10,000 pairs of the CD and the STD of CD are generated using Metropolis-Hastings algorithm  An independent bivariate normal distribution is used as a proposal distribution  MLE of the posterior distribution is used as a starting point CD from the fall time data Helicopter1Helicopter2Helicopter3 Sample mean of CDs Sample STD of CDs MLE of the CD mean MLE of the CD STD

Structural & Multidisciplinary Optimization Group 8/25 Problem3: Comparing predicted variability using posterior and observed variability  Handling the epistemic uncertainty due to finite sample  How to handle epistemic uncertainty in the CD and the test standard deviation estimates?  Comparing the posterior predictive distribution of the fall time and the empirical CDF of tests (combining epistemic and aleatory uncertainties)  Using p-box with 95% confidence interval of epistemic uncertainty (separating epistemic and alreatory uncertainties)

Structural & Multidisciplinary Optimization Group 9/25 Problem3: Comparing predicted variability using posterior and observed variability  Area metric of the posterior predictive distribution of CD CD from the fall time data Helicopter1Helicopter2Helicopter in 2 clips (ref) Sample mean of CDs Sample STD of CDs

Structural & Multidisciplinary Optimization Group 10/25 Problem3: Comparing predicted variability using posterior and observed variability  Area metric of the posterior predictive distribution of CD CD from the fall time data Helicopter1Helicopter2Helicopter in 2 clips (ref) Sample mean of CDs Sample STD of CDs

Structural & Multidisciplinary Optimization Group 11/25 Problem4: Predictive validation for the same height and different weight  Area metric of the posterior predictive distribution of CD CD from the fall time data Helicopter1Helicopter2Helicopter in 1 clips Sample mean of CDs Sample STD of CDs in 2 clips (ref) Sample mean of CDs Sample STD of CDs

Structural & Multidisciplinary Optimization Group 12/25 Problem4: Predictive validation for the same height and different weight  Area metric of the distribution of CD with p-box CD from the fall time data Helicopter1Helicopter2Helicopter in 1 clips Sample mean of CDs Sample STD of CDs in 2 clips (ref) Sample mean of CDs Sample STD of CDs

Structural & Multidisciplinary Optimization Group 13/25 Problem6: Predictive validation for different height and the same weight  Area metric of the posterior predictive distribution of CD CD from the fall time data Helicopter1Helicopter2Helicopter in 2 clips Sample mean of CDs Sample STD of CDs in 2 clips (ref) Sample mean of CDs Sample STD of CDs

Structural & Multidisciplinary Optimization Group 14/25 Problem6: Predictive validation for different height and the same weight  Area metric of the distribution of CD with p-box CD from the fall time data Helicopter1Helicopter2Helicopter in 2 clips Sample mean of CDs Sample STD of CDs in 2 clips (ref) Sample mean of CDs Sample STD of CDs

Structural & Multidisciplinary Optimization Group 15/25 Problem5: Linear model  Area metric with the prior  Data set CD from the fall time data Helicopter1Helicopter2Helicopter3 Sample mean of CDs Sample STD of CDs Prior Mean of CD = 1 / STD of CD = 0.28

Structural & Multidisciplinary Optimization Group 16/25 Comparison to predictive validation  Area metric of the posterior predictive distribution of CD  The predictive validation with the linear model is not as successful as that with the quadratic model  Area metric with p-box  Area metric with p-box tries to capture the extreme discrepancy between the predicted variability and the observed variability Helicopter1Helicopter2Helicopter in / 1 clips in / 2 clips (ref) Helicopter1Helicopter2Helicopter in / 1 clips in / 2 clips (ref)

Structural & Multidisciplinary Optimization Group 17/25 Problem5: Linear model  Area metric of the posterior predictive distribution of CD CD from the fall time data Helicopter1Helicopter2Helicopter in 2 clips (ref) Sample mean of CDs Sample STD of CDs

Structural & Multidisciplinary Optimization Group 18/25 Problem5: Linear model  Area metric of the distribution of CD with p-box CD from the fall time data Helicopter1Helicopter2Helicopter in 2 clips (ref) Sample mean of CDs Sample STD of CDs

Structural & Multidisciplinary Optimization Group 19/25 Problem5: Linear model with one clip  Area metric of the posterior predictive distribution of CD CD from the fall time data Helicopter1Helicopter2Helicopter in 1 clips Sample mean of CDs Sample STD of CDs in 2 clips (ref) Sample mean of CDs Sample STD of CDs

Structural & Multidisciplinary Optimization Group 20/25 Problem5: Linear model with one clip  Area metric of the distribution of CD with p-box CD from the fall time data Helicopter1Helicopter2Helicopter in 1 clips Sample mean of CDs Sample STD of CDs in 2 clips (ref) Sample mean of CDs Sample STD of CDs

Structural & Multidisciplinary Optimization Group 21/25 Comparison between quadratic and linear models  Area metric of the posterior predictive distribution of CD  Area metric of the distribution of CD with p-box Helicopter 1 Helicopter 2 Helicopter in / 1 clipLinear Quadratic in / 2 clips (ref) Linear Quadratic Helicopter 1 Helicopter 2 Helicopter in / 1 clipLinear Quadratic in / 2 clips (ref) Linear Quadratic

Structural & Multidisciplinary Optimization Group 22/25 Concluding remarks  Predictive validation for both quadratic and linear models  The predictive validation for different mass is a partially success  The predictive validation for different height is a success but the assumption of constant CD is not clearly proven  Comparison between models  Cannot conclude  Overall  Reason for the differences in the area metric is not clear  The effect of the manufacturing uncertainty is significant (i.e. very different area metrics for the same test condition)

Paper Clips HeightCalibrated?Model Area Metric Set 1 Model 1Set 1 Model 2Set 1 Model 3Set 3 Model 1Set 3 Model 2Set 3 Model 3 21noquadratic yesquadratic yesquadratic nolinear yeslinear yeslinear yesquadratic Kaitlin Harris, VVUQ Fall 2013 Comments: Chose to maintain uniform distribution for calibration parameter based on histogram results (vs normal) Conclusions: Best models: calibrated quadratic at both heights and calibrated linear with 2 paper clips Worst models: un- calibrated linear with 2 paper clips and un- calibrated linear with 1 clip for data set 1

Validation of analytical model used to predict fall time for Paper Helicopter By Nikhil Londhe *Calibrated Analytical Model is validated to represent experimental data *Quadratic dependence is valid assumption between drag force and speed *For given difference in fall height, Cd can be assumed to be constant Comparison of Analytical Cdf and Empirical Cdf Data Set 5, H=18.832, No. of Pins=2Helicopter 1Helicopter 2Helicopter 3 Validation Area Metric Before Calibration After Calibration Calibration Results Helicopter 1Helicopter 2Helicopter 3 Maximum Likelihood Estimate of Cd Standard deviation in Posterior pdf of Cd Predictive Validation for Data Set 5, No. of Pins =1 H=18.832ft.Helicopter 1Helicopter 2Helicopter 3 Validation Area Metric Validation Area Metric for Linear Dependence Model Data Set 5Helicopter 1Helicopter 2Helicopter 3 Validation Area Metric Validation of Cd is constant at different height, h=11.482ft Data Set 5, Pins = 2Helicopter 1Helicopter 2Helicopter 3 Validation Area Metric

Quadratic Dependence 2 clips Different height. Quadratic Dependence 1 clip Predictive Validation. Linear Dependence 2 clips & 1clip Prior VS. Posterior Dist. Quadratic Dependence 2 clips Prior VS. Posterior Dist. Course Project: Validation of Drag Coefficient -Yiming Zhang Validation based on 1 set of data: Validation based on 3 set of data: Summary: (1) Quadratic dependence seems accurate using one set; Quadratic and linear dependence both don’t match well using 3 sets; Prior Area Metric: Post Area Metric: Prior Area Metric: Post Area Metric: 0.2 Validation Area Metric: Validation Area Metric:  2 clips:  1 clip: Posterior Area Metric: Validation Area Metric:  Summary: Two confi. interval of Cd don’t coincide. Linear dependence seems inaccurate.  2 clips: Posterior Area Metric:  1 clip: Validation Area Metric:  Summary: Seems reasonable, but not accurate Validation Area Metric: Validation Area Metric: (2) SRQ is required to be fall time. If use Cd as SRQ, comparison could be more consistent and clear; (3) 0.8 seems a reasonable estimation of Cd. This estimation would be more accurate while introducing more sets of data.

Backup Slides

Structural & Multidisciplinary Optimization Group 27/25 Problems  Problem1: Conservative estimate of the fall time  Problem2: Comparing predicted variability and observed variability using prior  Problem3: Comparing predicted variability and observed variability using posterior  Problem4: Predictive validation for the same height and different weight  Problem5: Comparing the quadratic and linear models  Problem6: Predictive validation for different height and the same weight (proving the assumption of constant CD)

Structural & Multidisciplinary Optimization Group 28/25 Problem1: Conservative estimate of the fall time  Estimating the 5 th percentile of the fall time of one helicopter  Since fall time follows a normal distribution, estimating the 5 th percentile is based on estimating the mean and standard deviation (STD) of the fall time distribution  The mean and STD are estimated based on 10 samples  There is epistemic uncertainty in the estimated mean and STD due to a finite number of samples  To compensate the epistemic uncertainty, a conservative measure to compensate the epistemic uncertainty is required  Estimating the 5 th percentile with 95% confidence level