1. NCAR, Mar 9, 2004NNs for NCAR LWR1 Results with Neural Network Approximation for the NCAR CAM Long Wave Radiation Parameterization Vladimir Krasnopolsky, University of Maryland & NOAA/NCEP and Michael Fox-Rabinovitz, University of Maryland Acknowledgments: William Collins, Phillip Rasch, Joseph Tribbia, Dmitry Chalikov

2. NCAR, Mar 9, 2004NNs for NCAR LWR2 OUTLINE INTRODUCTION: Motivation for the study: new paradigm – combining deterministic GCM & statistical MLT (Machine Learning Techniques) APPROACH: “NeuroPhysics” - NN Approximations for Model Physics NCAR CAM-2 LWR: Accuracy and Performance of NN Approximation Comparison of CAM-2 Climate Simulations: two 10 Year Parallel Runs with the original LWR and its NN approximation CONCLUSIONS & DISCUSSION

3. NCAR, Mar 9, 2004NNs for NCAR LWR3 Distribution of Total Model Calculation Time Physics vs. Dynamics (in %) CAM-2 (T42 x L26):  3  x 3 

4. NCAR, Mar 9, 2004NNs for NCAR LWR4 Motivations for Applying NN to LWR Calculations of model physics take about 70% of total time in CAM-2 Calculations of radiation, LW & SW, takes about 60% of model physics calculations All current improvements of CAM (and other GCMs) such as increasing resolution result in further increase of calculation time for physics including LWR (to 90+% for physics) Reexamination of model physics calculations seems to be a timely and urgent problem!

5. NCAR, Mar 9, 2004NNs for NCAR LWR5 Generic Solution – “NeuroPhysics” Fast and Accurate NN Approximation for Parameterizations of Physics Learning from Data GCM X Y Parameterization F X Y NN Approximation F NN Training Set …, {X i, Y i }, …  X i  D phys NN Approximation F NN

6. NCAR, Mar 9, 2004NNs for NCAR LWR6 Major Advantages of NNs Relevant for Approximating Model Physics: NNs are very generic, accurate and convenient mathematical (statistical) models which are able to approximate model physics as complicated nonlinear input/output relationships (continuous mappings ). NNs are robust with respect to random noise and fault- tolerant. NNs are analytically differentiable (training, error and sensitivity analyses): almost free Jacobian! NNs are simple and fast. Training (only one for a model version!) is time consuming, application is NOT! NNs are well-suited for parallel processing

7. NCAR, Mar 9, 2004NNs for NCAR LWR7 NNs for NCAR CAM-2 Long Wave Radiation Parameterization

8. NCAR, Mar 9, 2004NNs for NCAR LWR8 NN for CAM-2 Physics CAM-2 Long Wave Radiation Long Wave Radiative Transfer: Absorptivity & Emissivity (optical properties):

9. NCAR, Mar 9, 2004NNs for NCAR LWR9 Neural Network for NCAR LW Radiation NN characteristics 220 Inputs: Profiles: temperature; humidity; ozone, methane, cfc11, cfc12, & N 2 O mixing ratios, pressure, cloudiness, emissivity Relevant surface characteristics: surface pressure, upward LW flux on a surface - flwupcgs 33 Outputs: Profile of heating rates (26) 7 LW radiation fluxes: flns, flnt, flut, flnsc, flntc, flutc, flwds Hidden Layer: One layer with 90 neurons (may go up to 150-200) Training: Training Data Set: Subset of about 100,000 instantaneous profiles simulated by CAM-2 for the 1-st year Training time: about 10 days (SGI workstation) Training iterations: 4,653 Validation on Independent Data: Validation Data Set (independent data): about 100,000 instantaneous profiles simulated by CAM-2 for the 2-nd year

10. NCAR, Mar 9, 2004NNs for NCAR LWR10 NN Approximation Accuracy and Performance vs. Original Parameterization Comparisons with ECMWF NN 1) ParameterModelBiasRMSEMean  Performance HR (  K/day) ECMWF 0.20.45  8 times faster NCAR Clear / Total 2. 10 -5 / 3. 10 -5 0.15 / 0.38 -1.401.98  80 times faster OLR (W/m 2 ) ECMWF 0.81.9 NCAR 0.011.7240.546.9 1) ECMWF NN approximation consists of the battery of 40 NNs Operational at ECMWF since October 2003

11. NCAR, Mar 9, 2004NNs for NCAR LWR11 Errors and Variability Profiles Bias - dashed RMSE - Solid Bias and RMSE profiles in  Bias and RMSE Profiles in K/day Bias = 3.  10 -5 K/day RMSE = 0.38 K/day PRMSE = 0.29 K/day  PRMSE = 0.25 K/day

12. NCAR, Mar 9, 2004NNs for NCAR LWR12 Errors and Variability Profiles (NASA) Bias and RMSE profiles in  Bias and RMSE Profiles in K/day Statistics For HRs Orig. vs. NN K/day Bias 2.  10 -4 6.  10 -4 RMSE 0.62 0.47 PRMSE 0.52 0.38  PRMSE 0.34 0.27

13. NCAR, Mar 9, 2004NNs for NCAR LWR13 NN Approximation Accuracy: Typical HR in K/d NN HR K/d Bias in  RMSE in  HR in K/d Error Distribution Error in K/d Level Statistics For HRs Orig. K/day NN K/day Min Val.-7.7-8.2 Max Val.6.15.2 Mean-0.779-0.778  0.270.26 Bias- 1.  10 -3 RMSE-0.145 P  626 mb

14. NCAR, Mar 9, 2004NNs for NCAR LWR14 Zonal Mean – Approximation Bias -0.1 0. 0.05 0.1 -0.05 K/day

15. NCAR, Mar 9, 2004NNs for NCAR LWR15 PRMSE = 0.47 & 0.34 K/day Individual Profiles PRMSE = 0.18 & 0.10 K/day PRMSE = 0.16 & 0.08 K/day Black – Original Parameterization Red – NN with 90 neurons Blue – NN with 150 neurons

16. NCAR, Mar 9, 2004NNs for NCAR LWR16 PRMSE = 0.14 & 0.07 K/day Individual Profiles PRMSE = 0.11 & 0.06 K/day PRMSE = 0.05 & 0.04 K/day Black – Original Parameterization Red – NN with 90 neurons Blue – NN with 150 neurons

17. NCAR, Mar 9, 2004NNs for NCAR LWR17 NCAR CAM-2: 10 YEAR EXPERIMENTS CONTROL: the standard NCAR CAM version (available from the CCSM web site) with the original Long-Wave Radiation (LWR) (e.g. Collins, JAS, v. 58, pp. 3224-3242, 2001) LWR/NN: the NCAR CAM version with NN approximation of the LWR (Krasnopolsky, Fox- Rabinovitz, and Chalikov, to be submitted in March 2004; and Fox-Rabinovitz, Krasnopolsky, and Chalikov, to be submitted in April 2004)

18. NCAR, Mar 9, 2004NNs for NCAR LWR18 CONSERVATION PROPERTIES (Global Annual Means) ParameterDifferences (original LWR parameterization - NN Approximation) in % Mean Sea Level Pressure (hPa) 0.001 Temperature0.002 Surface Temperature (  K) 0.03 Total Precipitation (mm/day) 0.3 Outgoing LWR – OLR (Wt/m 2 ) 0.3 Total Cloudiness (fractions 0.1 to 1.) 1.

19. NCAR, Mar 9, 2004NNs for NCAR LWR19 Zonal Mean Vertical Distributions and Differences Between the Experiments

20. NCAR, Mar 9, 2004NNs for NCAR LWR20 NCAR CAM-2 Zonal Mean Heating Rates 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b), contour = 0.1  K/day all in  K/day

21. NCAR, Mar 9, 2004NNs for NCAR LWR21 NCAR CAM-2 Zonal Mean U 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b), contour 1 m/sec all in m/sec

22. NCAR, Mar 9, 2004NNs for NCAR LWR22 NCAR CAM-2 Zonal Mean V 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b), contour 0.1 m/sec all in m/sec

23. NCAR, Mar 9, 2004NNs for NCAR LWR23 NCAR CAM-2 Zonal Mean Temperature 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b), contour 1  K all in  K

24. NCAR, Mar 9, 2004NNs for NCAR LWR24 NCAR CAM-2 Zonal RMSE for Heating Rates 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- RMS Difference (a) – (b), contour 0.1  K/day all in  K/day

25. NCAR, Mar 9, 2004NNs for NCAR LWR25 NCAR CAM-2 Zonal RMSE for Temperature 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- RMS Difference (a) – (b), contour 1  K all in  K

26. NCAR, Mar 9, 2004NNs for NCAR LWR26 Horizontal Distributions of Model Diagnostics

27. NCAR, Mar 9, 2004NNs for NCAR LWR27 NCAR CAM-2 Surface Pressure 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b), all in hPa MeanMinMax (a) 1011.48-3.912-0.871 (b) 1011.42- 3.766-0.926 (c) -0. 06-2. 311.87

28. NCAR, Mar 9, 2004NNs for NCAR LWR28 NCAR CAM-2 Total Cloudiness 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b), all in fractions MeanMinMax (a) 0.6070.0790.950 (b) 0.6140.0700.952 (c) 0.007-0.0850.086

29. NCAR, Mar 9, 2004NNs for NCAR LWR29 NCAR CAM-2 Total Precipitation 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b), all in mm/day MeanMinMax (a) 2.2750.02215.213 (b) 2.2880.02914.578 (c) 0.121-1.1891.184

30. NCAR, Mar 9, 2004NNs for NCAR LWR30 NCAR CAM-2 FLNT(OLR) 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b) all in W/m 2 MeanMinMax (a) 234.43133.62299.50 (b) 236.73136.69301.60 (c) 2.29-6.1011.94

31. NCAR, Mar 9, 2004NNs for NCAR LWR31 NCAR CAM-2 LWR Heating Rates (near 850 hPa) 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b) all in  K/day MeanMinMax (a) -1.698-3.912-0.871 (b) -1.697- 3.766-0.926 (c) -0. 001-0. 4180.615

32. NCAR, Mar 9, 2004NNs for NCAR LWR32 Horizontal Distributions of Model Prognostics

33. NCAR, Mar 9, 2004NNs for NCAR LWR33 NCAR CAM-2 Temperature (near 850 hPa) 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b) all in  K MeanMinMax (a) 251.08231.84298.24 (b) 250.99232.47298.10 (c) -0.09-1.050.81

34. NCAR, Mar 9, 2004NNs for NCAR LWR34 NCAR CAM-2 Temperature (near 200 hPa) 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b) all in  K MeanMinMax (a) 213.87231.84298.24 (b) 213.97232.47298.10 (c) 0.11-0.782.45

35. NCAR, Mar 9, 2004NNs for NCAR LWR35 NCAR CAM-2 U (near 850 hPa) 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b) all in m/sec MeanMinMax (a) 0.64-13.5617.74 (b) 0.58-13.0417.25 (c) -0.06-1.461.29

36. NCAR, Mar 9, 2004NNs for NCAR LWR36 NCAR CAM-2 U (near 200 hPa) 10 Year Average (a)– Original LWR Parameterization (b)- NN Approximation (c)- Difference (a) – (b) all in m/sec MeanMinMax (a) 16.35-14.5745.18 (b) 15.87-14.6044.51 (c) -0.48-2.712.17

37. NCAR, Mar 9, 2004NNs for NCAR LWR37 CONCLUSIONS: The proof of concept: Application of MLT/ NN for fast and accurate approximation of model physics has been successfully demonstrated for the NCAR CAM LWR parameterization. NN approximation of the NCAR CAM LWR is 80 times faster and very close to the original LWR parameterization The simulated diagnostic and prognostic fields are very close for the parallel NCAR CAM climate runs with NN approximation and the original LWR parameterization The conservation properties are very well preserved A solid scientific foundation is laid for development of MLT/NN approximations for other NCAR CAM physics components or a complete set of MLT/”Neuro- Physics”. Such a focused effort will result in a complete reexamination of the model physics calculations.

38. NCAR, Mar 9, 2004NNs for NCAR LWR38 OTHER POTENTIAL FUTURE DEVELOPMENTS The effort can be extended in the future to reexamination of calculations for: other CCSM components; and chemical and physical components of chemistry transport models MLT/NN can be applied for future development of new more sophisticated and time consuming parameterizations and even superparameterizations MLT/SVM approximations should be explored and compared to those of NN A feasibility study on MLT approximation of model dynamics will be conducted Developing MLT approximations for other time- consuming “bottleneck” procedures (e.g., solvers, iterations, inversions, transformations, etc.)