1. Short (Few-day) Simulations for Efficient Model Evaluation, Tuning and Calibration: Strategy, Framework and Early Results Atmosphere Group, Short Simulations Task Team (Yun Qian, Hui Wan, Phil Rasch, Wuyin Lin, and Shaocheng Xie) ACME All-Hands Meeting, May 5, 2015

2. ACME Needs New, Efficient Strategies for Model Evaluation and Tuning High-resolution, multi-decade simulations are hugely expensive Atmosphere Group, Short Simulations Task Team Single, long simulation Short, machine-capability jobs at LCF’s See also a poster by Matthew Norman et al.: “Computational Benefits of an Ensemble-Based Approach to Climate Modeling and Testing at Scale”

3. Previous Successes Fast processes, especially those related to clouds, are a major source of biases in current climate models SciDAC Multiscale Ensemble Average at day 3 % 5-yr Average % Total Cloud Fraction Difference, CAM5 4-min minus 30-min Time Step Wan et al. (2014) CAPT and Transpose-AMIP Global Mean Total Cloud Fraction in July 2009 Hindcasts Ma et al. (2014) Day 1 Day 2 Atmosphere Group, Short Simulations Task Team

4. Short Simulations Task Explore few-day simulations for model tuning and sensitivity studies Two-phase investigation –Parametric sensitivity experiments –Automatic parameter tuning Extensive use of UQ techniques –Sensitivity analysis Qian et al. (2015), Guo et al. (2014, 2015), Zhao et al. (2013) –Model calibration and auto-tuning Yang et al. (2012, 2013, 2014), Zou et al. (2014) Focus Region: the GPCI Cross-section Atmosphere Group, Short Simulations Task Team Figure courtesy of Heng Xiao (PNNL) JJA Cloud Fraction (2006-2010)

5. Accomplishments (Q1-Q3) A framework for short-ensemble-based parametric sensitivity experiments 31x128 CAPT hindcasts for July 2008 –1 degree resolution (ne30) –Using the multi-instance capability for simulation bundling –Finished within 3 days(!) on Titan Parametric sensitivity analysis –6 uncertain parameters related to turbulence and shallow convection –Quasi Monte Carlo method for sampling parameter space –Surrogate model for parametric sensitivity analysis Quasi Monte Carlo Sampling Atmosphere Group, Short Simulations Task Team

6. Dependence of Model Sensitivity on Cloud Regime Parametric Sensitivity of Shortwave Cloud Forcing Latitude (Degree North) Time Evolution Model Time (Day) Shallow Convection Relative Contribution to Total Variance Atmosphere Group, Short Simulations Task Team Time Evolution Model Time (Day) Shallow Convection Model Time (Day) Deep Convection

7. Next Steps How short is “short”? Answer depends on the science question and physical processes. (See, e.g., poster by Matthew Norman et al.) The “short ensemble” strategy can be applied to slower processes and model components Q4 and Q5 –Additional experiments and in-depth analysis of parametric sensitivity –Initial preparation for case studies of automatic parameter tuning Additional Comments Atmosphere Group, Short Simulations Task Team Our Posters Short simulations for efficient model evaluation, tuning and calibration, Part I: strategy and framework (Qian et al.) Part II: early results (Wan et al.)