1. 1 Pitfalls, Uncertainty, and Opportunities in Climate Change Science: NASA’s System Engineering for Decision Support University of Nebraska October 24-26, 2007 U.S. Geological Survey L. DeWayne Cecil Chief for Science Applications

2. 2 System Diagram for NASA’s Applied Sciences Activities Societal Benefits Crosscutting Solutions National Applications Research and Analysis Program Applied Sciences Program Operations Solutions Network Evaluation Water and Energy Climate Weather Carbon Solid Earth Atmospheric Composition Solar Verification and Validation Benchmarking supply demand R&O Integrated System Solution NASA Earth Science Research Rapid Prototyping Capacity Uncertainty Analysis, Scientific Rigor, Community Peer Review

3. 3 Research to Operations and Decision Support; Advantage Operations! Department of Interior Advantage; Development of Decision Support Systems and Tools by Charge Perhaps we need a DOI Climate Change Science Central, Analogous to Commerce's NCDC, Ashville, NC Alternative Approaches Larger Portion of R&A Portfolio becomes Applications UC-Boulder is Good Example NOAA’s RISA Program

4. 4 Data RAPID PROTOTYPING V&V DECISION SUPPORT TOOLS Observations VALUE & BENEFITS EARTH SYSTEM MODELS AND DATASETS EARTH OBSERVATIONS Predictions *Next Generation Missions Atmosphere: Aura, TRMM, OCO, CALIPSO, CloudSat, GPM, Aquarius Land : ICESat, MODIS Field Mission: Watershed- scale airborne campaigns, Ground-based monitoring network Watershed scale, 2-D, snow, ice, and water mass balance model (Plummer and Phillips, 2005) with input from NASA global scale projections Rigorous large ensemble probability distribution analyses OSSE datasets for next generation satellites Rigorous statistical design built into OSSEs upfront (NIST as a partner) Climate Models in ESMF: GISS Model E and other GMAO Analyses Bringing Global Climate Change Model Projections to the Watershed Scale: Pitfalls, Opportunities, and Uncertainties for Decision Support Natural & anthropogenic aerosols, black carbon Trace gas profiles Climate-Change Parameters Tropical/Global/Regional Precipitation Total Aerosols Use OSSE simulated next generation and current mission datasets for climate change scenario assessments WITH associated uncertainties carried throughout projections Use OSSE simulated data from next generation missions with existing measurements of climate change parameters from space to estimate the watershed-scale mass balance and climate change impacts Interagency Alignment: CCSP, CCTP, US GEO Uncertainty analyses, uncertainty analyses, Uncertainty analyses! Global/Regional/Water- shed scale model products Regional differences in aerosols and trace gas concentrations and impacts on climatology 12 – 18 month seasonal forecasts, 5 – 20 year projections, & Century timescale projections Impacts of global climate change on the Watershed scale Water resource management on local scale Climate-change impacts on waste- management facility sighting Decision support with uncertainty quantified and communicated (NSF as a partner) Uncertainty Analysis, Scientific Rigor, Community Peer Review Potential Partners

5. 5 Bringing Global Climate Change Model Projections to the Watershed Scale: Pitfalls, Opportunities, and Uncertainties for Decision Support Watershed-Scale Applied Science Questions? (1) How can global predictions of the effects of future rapid climate change and variability be enhanced and used? (2) How are uncertainties in global projections compounded, or not, at the watershed scale? (3) What are the implications for decision- support? Beluka Glacier, Siberia, Russia Upper Fremont Glacier, Wyoming, USA Øksfjordjøkelen, Norway

6. 6 Tools to Address the Science Questions Datasets Generated From Future-Mission OSSEs OCO, CALIPSO, CloudSat, GPM, Aquarius Datasets From Current Missions Aura, TRMM, ICESat, MODIS Global Climate Change Model Projections Using All Data GISS E, other GMAO Analyses Global Model Output Used As Watershed-Scale Model Input ESMF Compliant NASA Model Codes Partner’s ESMF Compliant Model Codes Uncertainty Analyses of Inputs and Outputs Project Columbia Inputs/Outputs MAP 2005 Inputs/Outputs MAP 2006 fvGCM (GEOS4) OSSE Datasets Tools for Uncertainty Analysis JCSDA GEOS-5 Decision Support Watershed Scale MB MODEL

7. 7 Example Statistical Tools with Potential to Enhance OSSE Analysis/Interpretation and For Analyzing Uncertainty of Model Projections Statistical Analyses Built Into Experiments Up Front Allow efficient, unambiguous comparisons of different methods for using data, including interactions between factors Uncertainty analysis Propagation of uncertainty in model inputs and outputs Necessary for realistic interpretation of all types of measurement results Computer intensive methods such as the Bootstrap and Bayesian analyses using Markov Chain Monte Carlo have the potential to handle complicated, intricate model projections and datasets Augment rigorous large ensemble probability distribution determinations

8. 8 Visualization of Experimental Results for Decision Support WHERE DID WE GO WRONG?

9. 9 2-D Visualization of The Effects of a 5 o Celsius Air-Temperature Suppression over 300 Years

10. 10 Late Pleistocene pluvial lakes in the Western Great Basin Reheis et al., 1989 Yucca Mtn., Nevada Why do We Care? What Can the Past Tell US?

11. 11 What are the Implications for Decision Support? Next Set of Questions for Decision Support How certain are we of the effects of the climate-change scenarios from global to watershed scale? What are the associated uncertainties, how are they communicated to the decision and policy makers, and how do they influence our decisions? What are the metrics for determining if we have provided the science data products from NASA’s current and next generation missions to enhance the decision making? Do we site our Nation’s mixed high-level radioactive and chemical waste repository at this location? If so, how do we include the climate-change scenario assessment in our decision making and in the design of our repository?

12. 12 Metrics for Determining Successes and Shortcomings of an Evolving Process Example Integrated System Solutions Metrics **Do Not Rely On One Metric Alone** Quantification and Communication of Uncertainty In Datasets In Climate-Change Projections (Forcings) In Decision Making Transparent and Inclusive Community of Practice Peer Review (This implies publication of results!) “Improved” Decision Support Expert analysts’ surveys are a significant first step, we must continue with more rigorous metrics, i.e., how did we enhance decision support, which combinatory factors/science products maximized enhancement?

13. 13 Sources of Uncertainty in Earth-system Science Uncertainty in Conceptual Model of System Watershed Scale Project Knowledge of Terrain Effects Changing Source of Storms and Precipitation Local-Regional-Global Drivers Uncertainty in Tools to Test Conceptual Model Observational Datasets Satellite CDRs (Climate Data Records), Long-Term, Traceable, Established Cal/Val, i.e. TSI, 30 year record In-situ Monitoring Network, Generally Short-Term, Cal/Val Is Underway Digital Models Incorrect concepts used to construct model code Who is checking this? Other Modelers! Suggest Independent Review Panels of Interdisciplinary Users, Decision Makers, and Data Providers as well as Modelers

14. 14 Sources of Uncertainty in Earth-system Science (continued) Applications Wrong Data for Input, Wrong Model Selected, Wrong Application of Model Scale is Wrong Temporal Division is Wrong Process (Weather to Climate) is Wrong

15. 15 Potential Partners to DOI and Sources of Experienced Researchers in Quantifying and Communicating Uncertainty for Decsion Support NIST Statistical Engineering Group Bootstrap Program, good for virtual datasets Bayesian Methods Monte Carlo Techniques National Science Foundation’s Decision Making Under Uncertainty in Climate Science (year 2+ of 5 year program) Climate Decision Center, Carnegie-Mellon University Decision Center for a Desert City, Arizona State U. Columbia University UC-Boulder RAND Potential Partners

16. 16 Contact Information L. DeWayne Cecil, Ph.D. USGS 900 N. Skyline Drive, Suite C Idaho Falls, ID 83402 208-528-2611 ldcecil@usgs.gov