1. Ronald M. Errico UMBC/GEST NASA/GMAO A plan for simulating current MW and IR radiance observations from the ECMWF Nature Run for the new “control” OSSE

2. Outline 1.Basic issues 2.Simulation of cloud-unaffected observation locations 3. Simulation of error-free, cloud-unaffected observations 4.Simulation of instrument and representativeness errors 5.Validation 6.Summary

3. 1. Basic issues

4. The basis for a plan In order to expedite production of a first new control OSSE, we set as a goal to produce an OSSE that validates significantly better than the previous OSSE did. The term “significantly better” is rather vague. Since it is difficult to guess how much effort would be required for various degrees of improvement, given our limited experience, we shall define this as we go along.

5. OSSE produced by Michiko et al. using T213L31 ECMWF T179 assimilation No radiances A fairly good match

6. OSSE produced by Michiko et al. using T213L31 ECMWF T179 assimilation Includes radiances A poorer match: Lets do better!

7. T(x,y,z) q(x,y,z) ε(x,y) surface T i,j qx i,j ε i,j Assimilation System Real System or Simulated from NR

9. Real vs. Assimilation-System Simulated Radiances

10. Two general approaches to simulating obs. from the NR 1. Simulate real observations as realistically as possible, so that the statistics of representativeness plus instrument errors are like those of the real assimilation system. a. Consider all physical aspects of the real problem. b. Consider all the sources of error and their characteristics. c. Validate by comparison with real error characteristics. 2.Simulate NR observations in an appropriately simple way and then add simulated errors by drawing from some PDF. The first approach requires much effort and, if consideration of a complex aspect is critical, the simulated error statistics may be unrealistic. On the other hand, with an incomplete characterization of present error statistics for the 2 nd approach, the PDFs used will likely be rather simple.

11. Issues: 1.Simulate realistic distributions of locations of accepted observations, with a realistic relationship with respect to the cloud distribution. 2. Simulate error-free, cloud-free radiances at those locations. 3.Simulate random additional errors due to instruments, spatial representativness, and forward models.

12. 2. Simulation of cloud-unaffected observation locations

13. Issue 1: Determine observation locations for accepted radiance observations In the case of IR observations, if there is a suggestion that the observation has been affected by clouds, it is discarded. This is part of the QC and/or data thinning/selection procedure. Similarly, in the case of MW observations, if there is a suggestion that the observation has been affected by precipitation, it is discarded. What remains after the discards is treated as though the RT for that observation (particular channel) has occurred in a column free of clouds or precipitation.

14. Our modest goal need not be to simulate the radiances from cloudy regions, but more simply to get the geographical distribution and selected innovation statistics “realistic.” This is simpler because most details regarding the clouds are irrelevant because we currently do not attempt to assimilate cloud-affected radiances. The presence of undetected thin clouds that can significantly affect representativeness error must also be considered.

15. HIRS Channel Jacobians Each channel is unaffected by clouds below some corresponding pressure level.

16. Determination of locations for simulated radiance observations for the Nature Run Goals: 1.Location of swaths to be like those for real observations 2.Counts and density of observations after thinning and QC to be nearly the same as in reality. 3.Observations to be rejected in regions with significant cloud cover in the NR. 4.Pattern of accepted observations should be indistinguishable from a collection of real distributions.

17. Accepted Observation Locations NOAA 17 HIRS Ch. 5 7/23/05 NOAA 17 HIRS Ch. 5 7/18/05 From Y. Zhu

18. Distribution of Accepted AIRS Obervations over 12-hour Period From Y. Zhu

19. frac(p<p_max) Determination of cloud-unaffected observation locations Horizontal grid box in NR data set Location of possible acceptable observation

20. Determination of cloud-unaffected observation locations The probability that an observation is unaffected by clouds present at an altitude where they would otherwise affect the RT depends principally on the fractional coverage, assuming that even thin clouds pose a problem. For each grid box where a satellite observation is given, use the cloud fraction to specify probability that it is a clear spot. Then use random number to specify whether pixel is clear. Use a functional relationship between probability and cloud fraction that we can tune to get a reasonable spatial distribution of accepted locations.

21. Sample functions for probability P=P(cloud fraction)

22. 1.This technique for determining cloud-unaffected observation locations can be tested and tuned off line (it only requires sets of real observation locations (accepted and not accepted) and NR cloud fractions and concentrations. 2.The probability can be defined using both cloud fraction and liquid water concentrations (large concentrations compensating for low fractions, if such combinations exist). 3. Begin by mapping cloud fractions and concentrations for a sample of individual days from the NR and looking at histograms of cloud fraction and perhaps scatter plots of fraction vs. concentration. 4.Perhaps it is sufficient to use simply aggregates of cloud information; e.g. for low, medium, and high clouds rather than for each model level. Strawman procedure for simulating observations

23. 3. Simulation of error-free, cloud-unaffected observations

24. Simulation of “error-free” radiances There are 2 choices: 1.Use the same RT model as for the assimilation system. In this case one source of representativeness error is absent and must be artificially added in later. This allows total control over the characteristics of that error, but it may be difficult to design such errors since those characteristics are not well known quantitatively. 2.Use a different RT model than that used by the assimilation system. This will introduce representativeness errors with rather complex characteristics, having correlations and a partially systematic nature. The implied error can also be unrealistically large, which would be difficult to correct.

25. Digression: The nature of representativeness error Components of the forward model or observation operator H H(x) = S {H * [I (x)]} I = Possible spatial and temporal interpolation operator H* = Physics model (RT model, precipitation scheme, etc.) S = Possible spatial or temporal integration operator (e.g., to obtain line of sight or footprint integrals or averages)

26. All these expressions will generally differ from each other, although not identically, even if x=z=x t at the points in common. Thus, a portion of rep. error is already implicitly introduced in the OSSE if the grid or RTM differ. Digression: The nature of representativeness error

27. Simulation of “error-free” radiances My recommendation: Use a different RT model than that used by the assimilation system. It may be easier to justify since then the character of the representativeness errors will not be so simple. We can first check offline that this does not yield rep. errors that are unrealistically large. This check is performed by simply computing the mean and variance of differences between radiances produced by two RTMs with identical T, q, p, etc. input. The variance should be less than the instrument plus rep. error specified in the DAS.

28. Simulation of “error-free” radiances At this point in their definition, these are unlike real observations in that they have no instrument error: they simulate the radiance field rather than the observations. This is because we have readily available estimates of the sum of the variances of the instrument and corresponding forward model errors (the observation error variance R specified in the DAS, but not both individually. A tuned sum of the two errors will therefore be added in a single and final step of the simulation procedure.

29. 4. Simulation of instrument and representativeness errors

30. Why is it important to simulate Rep. error well? 1.If R in the OSSE is like that in reality, then if the DAS and Nature Run models are as unalike as the DAS model and the atmosphere, the OSSE B will be like the real B. 2. If R and B in the OSSE are close to those in reality, then the DAS will be as optimal as it is in reality. 3. If 1 is satisfied, then analysis error statistics in the OSSE will be similar to those in reality, as will those for forecast error statistics. The OSSE will thus behave similar to the real system. 4. If the magnitude of R in the OSSE is significantly smaller than in reality, then the accuracies of both analysis and forecasts will tend to be overestimated.

32. Instrument Plus Representativeness Error 1.Since we have no real instrument, its errors must be entirely simulated. 2.If different radiative transfer models (more generally called “forward models” or “observation operators”) are used for simulation and assimilation, then a portion of representativness error has already been added. 3.The assimilation uses an interpolation algorithm (another form of forward model) for interpolating from fields defined at analysis grid points to values specified at observation locations. Similar models are also applied to the nature run for simulated observation locations. Since the assimilation and N.R. grids differ in resolution, there is another source for differing relationships between grid point values. For these reasons, another portion of representativness error has already been added when the simulated observations were created.

33. Simulating error as difference between 2 observation operators: ΔH(x) = H A (x) - H N (x) These errors are therefore have a systematic characteristic Simulating errors as a random variable: ΔH(x) = ε These errors are therefore have a stochastic characteristic Real errors ΔH(x) = H A (x) - H R (x) = ? These errors likely have both systematic and stochastic characteristics Different ways of simulating error

34. Strawman procedure for simulating instrument plus representativeness errors Note: 1 and 2 are not strictly necessary for this purpose since they only help in defining an initial iterate for R’. For interpreting results, however, they should be very helpful. Otherwise a starting value for R’ can be, e.g., R’=2R/3. 1.Obtain an estimate of the statistics of representativeness error due to using 2 different RT models (biases and variances) 2.Obtain an estimate of the statistics of representativeness error due to differing grid resolutions and bilinear interpolation 3.Generate errors to be added to each observation by drawing random numbers from a N(R’,0) distribution. Ignore biases since any large bias will be removed by the assimilation anyway. Use R’<R, R being the error value specified in the DAS. 4. Run the assimilation for a short time and note the variance of the innovations. Re-adjust R’ so that this variance better matches the corresponding variance computed for the real assimilation. 5.After 1 or 2 iterations of this, use the final R’ in any further experiments.

35. Error statistics to consider 1. Biases: My recommendation is simply set to 0 2. Variances: My recommendation is simply set to constant that is a function of channel only. 3. Correlations: My recommendation is to assume no correlation. 4. Distribution function: My recommendation is to use a Gaussian (since these are applied to only those simulated observations that are intended to pass QC). Note: Real errors are likely functions of geography, local flow, season, viewing angle, etc., and are mutually correlated. It is difficult to produce more realistic statistics without better guidance regarding what those statistics are. It may also be unnecessary.

36. 5. Validation

37. If an alternative procedure of using all the NR cloud information within the context of a cloudy RT scheme appears necessary, then it should also be necessary that many details regarding the clouds must be realistic in the NR, not just their gross distribution. Strawman procedure for simulating observations

38. Validation of simulated locations Ideally, the simulation of locations is validated, if global maps of simulated observation locations for individual observing periods are indistinguishable from random maps of real observation locations for equivalent periods.

39. OSSE produced by Michiko et al. using T213L31 ECMWF T179 assimilation Includes radiances A poorer match: Lets do better!

40. (J/Kg) …all observing systems provide total monthly benefit Impacts of various observing systems Totals GEOS-5 July 2005 Observation Count (millions) (J/Kg) NH observations SH observations From R. Gelaro & Y. Zhu

41. 6. Summary

42. 1.Obtain an estimate of the statistics of representativeness error due to using 2 different RT models (biases and variances) 2.Obtain an estimate of the statistics of representativeness error due to differing grid resolutions and bilinear interpolation 3.Examine spatial distribution of cloud fractions for H, M, and L clouds at some single times. Also look at some other measure of cloud concentration in a similar way. 4. It may be useful to construct a histogram of cloud fractions, say in bins of 0.1 units. Also, a scatter diagram of cloud fraction vs. cloud concentration. Some preliminary calculations

43. 1.For each observation location …… 2.Bilinearly interpolate cloud fractions from NR grid to location 3.Compute P(f, tuning parameter) 4.Select random number x from uniform distribution (0,1) 5.If x>P then consider cloud free for this height of cloud, otherwise cloud covered point 6.If cloud free, then bilinearly interpolate q,T to location and compute radiances for these unaffected channels 7.If cloudy region, set radiance to very small value such that QC will detect and discard. 8.Repeat this process with various tuning parameters for P until the observation count and distribution look reasonable. Then use this parameter for all further experiments. Strawman procedure for simulating observations

44. 2-D: Low cloud fractional coverage Medium cloud fractional coverage High cloud fractional coverage Total cloud cover Convective precipitation Stratiform precipitation 3-D Cloud liquid water Cloud ice water Cloud Related Nature Run Fields