1. Examination of Observation Impacts Derived from OSEs and Adjoint Models
Ron Gelaro and Yanqiu Zhu
NASA Global Modeling and Assimilation Office
Also: Ricardo Todling, Ron Errico

2. Outline of Talk
 Estimation of observation impact: Adjoint (ADJ) method
 ADJ impact results in NASA’s GEOS-5 DAS
 Comparison of ADJ and OSE results
 Combined use of ADJ and OSEs
 Concluding remarks

3. Estimating Observation Impact
Forecast error measure (dry energy, sfc–140 hPa):
Taylor expansion of change in due to change in :
Analysis equation allows transformation to observation-space:
3rd order approximation of in observation space:
…summed observation impact
analysis adjoint
model adjoint

4. Properties of the Impact Estimate
 The impact of arbitrary subsets of observations (e.g. instrument type, channel, location) can be easily quantified by summing only the terms involving the desired elements of
 The “weight” vector is computed only once, and involves the entire set of observations…removing or changing the properties of one observation changes the weight of all other observations.
 Valid forecast range limited by tangent linear assumption for
…the observation improves the forecast
…the observation degrades the forecast
…see Langland and Baker (2004), Errico (2007), Gelaro et al. (2007)

5. GEOS-5 Adjoint Observation Impact Experiments
Analysis System
 3DVAR Gridpoint Statistical Interpolation (GSI)
 0.5o resolution, 72 levels
 Adjoint: Exact line-by-line (Zhu and Gelaro 2007)
Forecast Model
 GEOS-5: FV-core + full physics
 0.5o resolution, 72 levels
 Adjoint: FV-core 1o resolution + simple dry physics
Experimentation
 6h data assimilation cycle, July 2005 and January 2006
 24h forecasts from 00UTC to assess observation impact
 Separate error response functions for the globe, NH, SH and tropics

6. Accuracy of Observation Impact Estimate
GEOS-5 July z
Orders of approximation
(true)
Error Reduction (J/kg)
Gelaro et al. 2007
 All values negative…observations provide benefit overall
 2nd and 3rd order approximations recover ~85% of true impact computed from model fields directly
 Accuracy of observation space estimate allows meaningful aggregation by observation type, location, channel, etc.

7. Total 24hr Forecast Error Reduction due to Observations
July UTC
(J/kg)
(J/kg)
GEOS-5 Adjoint Data Assimilation System

8. Total 24hr Forecast Error Reduction due to Observations
January UTC
(J/kg)
(J/kg)
GEOS-5 Adjoint Data Assimilation System

9. Impact of satellite observations by channel
July UTC
AIRS
improve
degrade
Channels
Global impact of most channels is beneficial on average…with some exceptions
-0.7
0.0
Forecast Error Reduction (J/kg)
AMSU-A
Channels
-7.0
0.0
Forecast Error Reduction (J/kg)
GEOS-5 Adjoint Data Assimilation System

10. Localized examination of AIRS impacts
July UTC
degrade
AIRS impact map
(All Channels)
H20
degrade
AIRS impact by channel
improve
(20-50N, 0-80E)

11. Fraction of Observations that Improve the Forecast
GEOS-5 July z
Control
AIRS
No AMSU-A
Control
AMSU-A
No AIRS
…only a small majority of the observations improve the forecast

12. GEOS-5 Observing System Experiments (OSEs)
 The following observation sets were individually excluded from the data assimilation system during July 2005 and January 2006:
all AMSU-A radiances from 1 satellite (N-16): no amsua1
all AMSU-A radiances from 2 satellites (N-15,16): no amsua2
all AMSU-A radiances from 3 satellites (N-15,16, Aqua): no amsua3
all AIRS radiances: no airs
all rawinsonde observations: no raob
all satellite winds (AMVs): no satwind
all aircraft observations: no aircraft
all scatterometer winds from QuikSCAT: no qkscat
 Control analysis used for verification, impact measured using the same energy metric as in the ADJ experiments for the globe, NH, SH and tropics

13. Forecast error growth in OSEs
July 2005 e e

14. Comparison and Interpretation of ADJ and OSE Results
…a few things to keep in mind…
 The ADJ measures the impacts of observations in the context of all other observations present in the assimilation system, while the OSE changes/degrades the system ( differs for each OSE member)
 The ADJ measures the impact of observations in each analysis cycle separately and against the control background, while the OSE measures the impact of removing information from both the background and analysis in a cumulative manner
 Comparison is restricted to the forecast range and metric for which the adjoint results are valid on the one hand (24h-energy in this study) and to the observing systems tested in the OSE on the other

15. ‘Direct’ quantitative comparison of ADJ and OSEs
Define the fractional impact of observing system for each approach:
 Measures the % decrease in error due to the presence of observing system with respect to the background forecast 
 Measures the % increase in error due to the removal of observing system with respect to the control forecast 

16. % Contributions to 24hr Forecast Error Reduction
January 2006

17. % Contributions to 24hr Forecast Error Reduction
July 2005

18. OSE Time Series of 24-hr Forecast Error Norm
January z SH Error Norm
Energy (J/kg)
July z SH Error Norm
Skill collapses when all AMSUA removed during SH winter…OSE and ADJ differ
Energy (J/kg)
Date

19. Normalized % Contributions to 24hr Forecast Error Reduction
January 2006
July 2005
…normalized results for the tropics (shown here) in much better agreement; results for the extratropics mostly unchanged (not shown)

20. Combined Use of ADJ and OSEs
 Both OSEs and ADJ measure the net effect of observations on the forecast
 We are also interested in dependencies and redundancies between observing systems are observations are added or removed
 Such information is implicitly available in an OSE in terms of the responses of the remaining observing systems when a given set of observations is removed.
 These responses can be measured through the combined use of OSEs and ADJs, by applying the ADJ to the perturbed (vs. only the control) members of an OSE

21. Combined Use of ADJ and OSEs
…ADJ applied to various OSE members to examine how the mix of observations influences their impacts
Same forward and adjoint systems as in slide 1.
Top panel: Both blue and red are good…more obs improve forecast than degrade. Gray not good…more than half of the obs degrade.
Bottom panel: Magnitude of impact. Note that scale is not linear. Negative (blue) is good. Positive (red) is bad. Sorry for the different color convention than in top panel.
Removal of AMSUA results in large increase in AIRS (and other) impacts
Removal of AIRS results in significant increase in AMSUA impact
Removal of raobs results in significant increase in AMSUA, aircraft and other impacts (but not AIRS)

22. Combined Use of ADJ and OSEs
…ADJ applied to various OSE members to examine how the mix of observations influences their impacts
Same forward and adjoint systems as in slide 1.
Top panel: Both blue and red are good…more obs improve forecast than degrade. Gray not good…more than half of the obs degrade.
Bottom panel: Magnitude of impact. Note that scale is not linear. Negative (blue) is good. Positive (red) is bad. Sorry for the different color convention than in top panel.
Removal of AMSUA results in large increase in AIRS impact in tropics
Removal of wind observations results in significant decrease in AIRS impact in tropics (in fact, AIRS degrades forecast without satwinds!)

23. Conclusions
 Despite fundamental differences in how impact is measured, ADJ and OSE methods provide comparable estimates of the overall ‘importance’ of most observing systems
Some differences in OSE and ADJ results should be expected and do not necessarily point to shortcomings either:
different treatment of background information
removal of whole observing systems that contribute disproportionately to analysis quality (AMSU-A)
Information gleaned from OSEs and ADJs should be viewed as complementary; ADJ extends, not replaces, OSEs:
applicable forecast range, metrics differ
ADJ well suited for routine monitoring
 The combined use of ADJ and OSEs illuminate the complex, complementary nature of how observations are used by the assimilation system