1. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 0 PROTECTION AGAINST MODELING AND SIMULATION UNCERTAINTIES IN DESIGN OPTIMIZATION NSF GRANT DMI-9979711 Bernard Grossman, William H. Mason, Layne T. Watson, Serhat Hosder, and Hongman Kim Virginia Polytechnic Institute and State University Blacksburg, VA Raphael T. Haftka University of Florida Gainesville, FL Life-Cycle Engineering Program Meeting and Review 24-25 September 2002 Albuquerque, NM

2. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 1 Research Study 1 (by Hongman Kim) Statistical Modeling of Optimization Errors due to Incomplete Convergence Detection and Repair of Poorly Converged Optimization Runs Research Study 2 (by Serhat Hosder) Observations on Computational Fluid Dynamics (CFD) Simulation Uncertainties Research Performed Under the Project

3. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 2 Research Study 1

4. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 3 Find error characteristics of a structural optimization of a high speed civil transport Estimate error level of the optimization procedure Identify probabilistic distribution model of the optimization error Estimate mean and standard deviation of errors without expensive accurate runs Improve response surface approximation against erroneous simulation runs via robust regression Objective

5. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 4 250 passenger aircraft, 5500 nm range, cruise at Mach 2.4 Take off gross weight (W TOGW ) is minimized Up to 29 configuration design variables including wing, nacelle and fuselage geometry, fuel weight, and flight altitude For this study, a simplified 5DV version is used High Speed Civil Transport (HSCT)

6. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 5  For Case 2, the initial design point was perturbed from that of Case 1, by random factors between 0.1 ~ 1.9  In average, Case 2 has the same level of error as Case 1  The errors were calculated with respect to higher fidelity runs with tightened convergence criteria Effects of initial design point on optimization error

7. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 6 To get optimization error we need to know true optimum, which is rarely known for practical engineering optimization Optimization error = OBJ* - OBJ* true To estimate OBJ* true Find convergence setting to achieve very accurate optimization This approach can be expensive Estimating error of incomplete optimization

8. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 7 Widely used in reliability models (e.g., lifetime of devices) Characterized by a shape parameter  and a scale parameter  PDF function according to  The Weibull distribution model

9. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 8 Errors of two optimization runs of different initial design point s = W 1 – W t t = W 2 – W t, (W t is unknown true optimum.) The difference of s and t is equal to W 1 – W 2 x = s – t = (W 1 – W t ) – (W 2 – W t ) = W 1 – W 2 We can fit distribution to x instead of s or t. s and t are independent Joint distribution of g(s) and h(t) g(s;  1 ) PDF s, t h(t;  2 ) x=s-t MLE fit for optimization difference x using f(x) No need to estimate W t Difference fit to estimate statistics of optimization errors

10. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 9 Difference fit is applied to the differences between Cases 1 and 2, assuming the Weibull model for both cases Even when only inaccurate optimizations are available, difference fit can give reasonable estimates of uncertainty of the optimization Comparison between error fit and difference fit

11. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 10 Another set of optimization with a difference initial design point was enough, which is straightforward and no more expensive The difference fit gave error statistics as accurate as the error fits involving expensive higher fidelity runs Estimated distribution parameters

12. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 11 Robust regression techniques Alternative to the least squares fit that may be greatly affected by a few very bad data (outliers) Iteratively reweighted least squares (IRLS) Inaccurate response surface approximation is another error source in design process  Improve response surface approximation by repairing outliers Outliers and robust regression

13. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 12 Weighted least squares: Initially all points have same weight of 1.0. At each iteration, the weight is reduced for points that are far from the response surface. This gradually moves the response surface away from outliers. IRLS approximation leaves out detected outliers 1 dimensional example Iteratively Reweighted Least Squares

14. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 13 Regular weighting functions Symmetrical Huber’s function gives non- zero weighting to big outliers Biweight rejects big outliers completely Nonsymmetric weighting function make use of one-sidedness of error by penalizing points of positive residual more severely Weighting functions of IRLS Nonsymmetric weighting function Aggressive search by reducing tuning constant B

15. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 14 Comparison of RS before and after repair Improvements of RS approximation by outlier repair

16. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 15 Conclusions Multiple simulation results enable statistical techniques to estimate the uncertainty level of the simulation error Weibull distribution successfully used to model the convergence error of the optimization runs The probabilistic model allowed us to estimate average errors without performing high fidelity optimizations IRLS and NIRLS were successfully used to detect the outliers originating from poorly converged optimizations Detection and repair of outliers pay off in better accuracy of response surfaces, which reduces the design uncertainties

17. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 16 Research Study 2

18. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 17 Motivation Drag polar results from 1 st AIAA Drag Prediction Workshop (Hemsch, 2001)

19. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 18 Objective Finding the magnitude of CFD simulation uncertainties that a well informed user may encounter and analyzing their sources We study 2-D, turbulent, transonic flow in a converging-diverging channel complex fluid dynamics problem affordable for making multiple runs known as “Sajben Transonic Diffuser” in CFD validation studies

20. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 19 Transonic Diffuser Problem P/P0i Strong shock case (Pe/P0i=0.72) Weak shock case (Pe/P0i=0.82) Separation bubble experiment CFD Contour variable: velocity magnitude streamlines

21. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 20 Uncertainty Sources (following Oberkampf and Blottner) Physical Modeling Uncertainty PDEs describing the flow Euler, Thin-Layer N-S, Full N-S, etc. boundary conditions and initial conditions geometry representation auxiliary physical models turbulence models, thermodynamic models, etc. Discretization Error Iterative Convergence Error Programming Errors We show that uncertainties from different sources interact

22. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 21 Computational Modeling General Aerodynamic Simulation Program (GASP) A commercial, Reynolds-averaged, 3-D, finite volume Navier-Stokes (N-S) code Has different solution and modeling options. An informed CFD user still “uncertain” about which one to choose For inviscid fluxes (commonly used options in CFD) Upwind-biased 3 rd order accurate Roe-Flux scheme Flux-limiters: Min-Mod and Van Albada Turbulence models (typical for turbulent flows) Spalart-Allmaras (Sp-Al) k-  (Wilcox, 1998 version) with Sarkar ’ s compressibility correction

23. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 22 Grids Used in the Computations Grid levelMesh Size (number of cells) 140 x 25 280 x 50 3160 x 100 4320 x 200 5640 x 400 A single solution on grid 5 requires approximately 1170 hours of total node CPU time on a SGI Origin2000 with six processors (10000 cycles) y/h t Grid 2 Grid 2 is the typical grid level used in CFD applications

24. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 23 Uncertainty in Nozzle Efficiency

25. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 24 Discretization Error by Richardson’s Extrapolation Turbulence model Pe/P0iestimate of p (observed order of accuracy) estimate of (n eff ) exact Grid level Discretization error (%) Sp-Al 0.72 (strong shock) 1.3220.71950 114.298 26.790 32.716 41.086 Sp-Al 0.82 (weak shock) 1.5780.81086 18.005 23.539 31.185 40.397 k-  0.82 (weak shock) 1.6560.82889 14.432 21.452 30.461 40.146 order of the method a measure of grid spacinggrid level error coefficient

26. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 25 Error in Geometry Representation

27. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 26 Downstream Boundary Condition

28. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 27 the total variation in nozzle efficiency 10%4% the difference between grid level 2 and grid level 4 6% (Sp-Al) 3.5% (Sp-Al) the relative uncertainty due to the selection of turbulence model 9% (grid 4) 2% (grid 2) the uncertainty due to the error in geometry representation 0.5% (grid 3, k-  ) 1.4% (grid 3, k-  ) the uncertainty due to the change in exit boundary location 0.8% (grid 3, Sp-Al) 1.1% (grid 2, Sp-Al) Uncertainty Comparison in Nozzle Efficiency Strong Shock Weak Shock Maximum value of

29. NSF Grant DMI-9979711 2002 NSF-SNL Grantees Meeting, 09/24/2002, Albuquerque NM 28 Conclusions Based on the results obtained from this study, Informed users may get large errors for the cases with strong shocks and substantial separation Grid convergence not achieved with grid levels that have moderate mesh sizes Uncertainties from different sources interact, especially in the simulation of flows with separation Discretization error and turbulence models are dominant sources of uncertainty in nozzle efficiency We should asses the contribution of CFD uncertainties to the MDO problems that include the simulation of complex flows