1. National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Prospects for Model-Based Flight Qualification of Spacecraft Dr. Lee D. Peterson Principal Engineer Mechanical Systems Division Jet Propulsion Laboratory USC Uncertainty Quantification Workshop 13 April 2009 © 2009 California Institute of Technology. Government sponsorship acknowledged.

2. 2 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology This talk summarizes technology that could enable model-based flight qualification for future missions What are some motivating mission needs? What technologies might be needed to achieve this? How are we meeting this challenge at JPL?

3. 3 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology What are some motivating mission needs?

4. 4 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology JPL is operated by the California Institute of Technology for NASA Founded in 1936 by a group of CalTech graduate students looking for a good place to test their rockets Home of America’s first satellite in 1958 (Explorer 1) Today responsible for a variety of robotic space science missions Planetary science Earth remote sensing science Astrophysics and astronomy

5. 5 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology JPL missions potentially face the challenge of extrapolation via models Large apertures Telescopes and RF antennae 0-g vs 1-g precision Electric propulsion Decade-long lifetime from year-long tests Planetary entry, descent, and landing systems Multi-disciplinary, multi- domain simulations Large solar array power systems Limited capability for thermal-vac testing MSL supersonic parachute simulation Mesh reflector in 25 chamber Common technological challenges: how to develop high- fidelity, validated and reliable models

6. 6 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology The need for predictive models and simulations extends throughout the space mission lifecycle Mission science Quantified uncertainties in science models for validating predictions Instrument Development Observing science simulation experiments with quantified uncertainties for the development of instrument error budgets, requirements, margins, cost and risk Engineering Design, Integration and Verification Predictive simulations of engineering designs with quantified margins and uncertainties for model-based flight qualification of spacecraft

7. 7 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Validated, Integrated System Model - Structures/Thermal/Optics/Control - Sub-nanometer resolution - Gravity effects (damping, hysteresis) One driving application: How do we validate a system that cannot be fully tested on the ground? Component or Subsystem Test Component or Subsystem Model Extrapolated 0-g performance - Closed-loop robustness - Validated error budget - System validation by analysis Verification from extrapolated ground test data is becoming a necessity for high performance missions Subsystem coupling and interaction makes this challenging

8. 8 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Some unique aspects of driving mission needs may be new to the UQ community Different physics Electric propulsion 1-g to 0-g microdynamics Picometer-scale optics milli-K thermal stability Different applications Multiple-domain meshes Highly coupled systems Actively controlled systems Different processes System engineering and model V&V Integration into flight practices Application over system life cycle The space community offers new challenges for uncertainty quantification research.

9. 9 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Multi-physics, integrated models are a challenge for model V&V Easy Single-discipline models Difficult Integrated models of “loosely coupled” sub-systems Separable sensitivity analyses allowed Can use “bucket brigade” approach (even for UQ) Hard Integrated models of “tightly coupled” sub-systems Sensitivities evaluated at the system level Requires truly integrated simulation

10. 10 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology What technologies might be needed to achieve this?

11. 11 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Current practice varies across the space industry, but some commonalities can be identified Margin policies are routine engineering practice Example: “MUF’s” (Model Uncertainty Factors) Determined from prior experience to account for both prediction uncertainty and design maturity Moderate scale Monte Carlo simulation might be used to assess prediction uncertainty Rigorous model verification and validation (MV&V) practices are not (yet) routine System Response Quantity Probability of Response MeanReqt Uncertainty MUF

12. 12 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Recent standards and recommended practices are also influencing NASA Large activities over the past 10-15 years Recent NASA Standard for Models and Simulation (NASA-STD-7009, released July 2008) In response to the Columbia Accident Investigation Board (CAIB) report Embodies much of the modern model V&V language NASA, AIAA, ASME, DOE and DOD Guidelines and Recommended Practices

13. 13 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology A key challenge is the issue of extrapolation beyond the validation domain system or environmental parameter physical and geometrical complexity Application Domain Validation Domain system or environmental parameter physical and geometrical complexity Application Domain Validation Domain system or environmental parameter physical and geometrical complexity Application Domain Validation Domain inference [Oberkampf et al, 2004] Extrapolation means the model gets the right answer … for the right reasons.

14. 14 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Model tuning (calibration) is a common misunderstanding of model V&V in aerospace “Model Calibration” “Model Tuning” “Model Update” By themselves, these do not “validate” the model (i.e. they do not quantify predictive capability) “Model Correlation” “Information Gap” theory says that model tuning can render a model less reliable for prediction outside the test domain.

15. 15 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology A hierarchical approach most naturally fits the usual spacecraft integration and test flow Unit Tests Benchmark Tests Component Tests System Tests Basic physics and empiricisms. Tightly controlled test conditions. Very low test uncertainty. Two or more basic physics Ideal boundary conditions Predictive evaluation of unit tests. More difficult to isolate single error source. Undesired for empiricism. Predictive evaluation of benchmark tests. Moderate to high uncertainty in test conditions. Not used for empiricism. Predictive evaluation only. Lower level validation test results might be reused in multiple projects.

16. 16 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Some example technologies might be needed to make model-based flight qualification practical (Presented in no particular order … ) Reduced order modeling (ROM) for UQ of large scale simulations Methods for efficient UQ for models that couple multiple, large scale commercial or proprietary simulation codes with incompatible meshes Methods for the roll-up of lower level benchmark or component level tests to subsystem and system uncertainties Methods for early life cycle (simple) system models that evolve into complex, coupled system models Development of “open source” databases with unit tests or benchmark tests

17. 17 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology More example technologies we might need (again in no particular order … ) Effective methods for treating epistemic uncertainty in large scale simulations User interfaces for preparation and execution of large scale UQ analyses, on either commercial or proprietary codes Methods for inverse statistical modeling or inverse epistemic modeling from test data Methods for extrapolation of margins and uncertainties outside the domain of model validation tests.

18. 18 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology How are we meeting this challenge at JPL?

19. 19 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Some new technologies under development at JPL address some key aspects of this challenge High performance integrated modeling software Uncertainty determined by other than computational limitations Simple models with large uncertainty early in life cycle Complex models with narrow uncertainty later How to evolve from one to the other in a seamless manner? How to incorporate test data in an evolving manner? Software framework to manage model development, UQ and model V&V throughout project life

20. 20 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Example: Cielo (JPL) has been coupled to DAKOTA (Sandia) for CEV UQ design analysis CEV TPS FEM: (LaRC, ARC) Carrier structure (CS) divided into 9 sandwich zones; 2 shoulder zones Uncertainty variables: CS Zone 1 face sheet thickness (0.020” nom.) FoS for load case (1.4 nom.) Gaussian distribution (5% variance) assumed in uncertainty variables Response variables: Through-thickness (TTT) and in-plane (IP) tile stresses computed DAKOTA used to quantify uncertainty in stress predictions IP max and ultimate stress results shown [J. Schiermeier, JPL 3542, 6/08] DAKOTA (Sandia) manages uncertainty analysis loop: Cielo (JPL) thermal- structural analysis: [slide from W. Wilkie, JPL 355L, 9/08] Common mesh for both thermal and structural analysis

21. 21 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology JPL engineers and scientists are collaborating to address these challenges Cross-Lab working group formed to define a strategy to address the advanced modeling and UQ challenge Three driving capabilities: Quantified uncertainties in science models for validating predictions Observing science simulation experiments (OSSE) with quantified uncertainties for development of instrument error budgets, requirements, margins, cost and risk Predictive simulations of engineering designs with quantified margins and uncertainties for model-based flight qualification of spacecraft

22. 22 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology This talk summarized some UQ R&D needs for future missions Variety of future missions will rely on models with quantified margins and uncertainties. Research in high performance computation and basic UQ methods are needed JPL is developing new tools and methods aimed at UQ for predictive space system models and simulations