1 Observation Impact Using a Variational Adjoint System PI: Dr. James Cummings, Code Co-PIs: Dr. Hans Ngodock, Dr. Alan Wallcraft, Dr. Scott Smith, Dr. Nancy Baker (code 7531) NRL 6.2 New Start Proposal 73-P Approved! Funding Cycle: Oct 2009 – Sep 2012

2 Develop a system for assessing the impact of any and all observations assimilated in Navy ocean forecast models. System must be computationally efficient and capable of being run in near-real-time for routine observation monitoring. Impacts of observation subsets must be easily quantifiable: instrument type (platform) measurement variable geographic region vertical depth level Objective: Routine Assessment of Data Impacts

3 Ocean predictions require very fine spatial scales: – model resolutions greatly exceed observation density – ocean models are not well constrained by observations – accurate forecasts on fine spatial scales depend on optimal use of the relatively sparse ocean observations All observations do not have equal value: – in terms of reducing forecasting error – how can we quantify the impact of each observation? Ocean observing and forecast systems are in continuous evolution: – how can we provide routine assessments of data impact? Motivation: Need for Optimal Use of Observations

4 Conventional observing system experiments (OSE): withhold (deny) observations from the assimilation shows impact on all forecast parameters difficult to systematically evaluate all observation data types computationally expensive – cannot be performed routinely removal (or addition) of an observation changes the analysis constraints on the remaining data this by itself can alter the outcome of the assimilation and the forecast Approach: Methods for Estimating Data Impacts Adjoint Technique shows impact on all observation data types equivalent to an OSE for short forecast periods computationally inexpensive – same cost as a single run of the forecast model and the data assimilation can easily be implemented in operations provides routine generation of data impact statistics

5 Observation (y) Data Assimilation System Forecast Model Forecast (x f ) Gradient of Cost Function J: (  J/  x f ) Background (x b ) Analysis (x a ) Adjoint of the Forecast Model Tangent Propagator Observation Sensitivity (  J/  y) Analysis Sensitivity (  J/  x a ) Observation Impact (  J/  y) Adjoint of the Data Assimilation System What is the impact of observations on measures of forecast error (J) ? Adjoint System Analysis – Forecast System

6 Observations move the forecast from the background trajectory (X b ) to the trajectory starting from the new analysis (X a ) “Observation impact” is the combined effect of all of the observations on the difference in forecast error (e f - e g ) Observation Impact Concept - OBSERVATIONS ASSIMILATED Forecast Error VERIFYING ANALYSIS Forecast Lead Time XgXg XfXf XbXb XaXa t=-24 t=0 t=24 ● ● ● ● ● egeg efef

7 Observation Impact Equation Langland and Baker (2004) Forecasts are made with HYCOM/NCOM using NCODA 3DVar/4DVar assimilation Adjoint versions of HYCOM/NCOM and NCODA 3DVar/4DVar are used to calculate the observation impact Impacts of observation subsets can be computed based on: instrument type, measurement variable, geographic region, etc Observations assimilatedAdjoint of assimilation Adjoint sensitivity gradients in model grid-point space Forecast error difference for each observation

8 Observation impact interpretation - < 0.0 the observation is BENEFICIAL > 0.0 the observation is NON-BENEFICIAL For any observation assimilated, if... forecast errors decrease – the effect of the observation is to make the error of the forecast started from x a less than the error of the forecast started from x b forecast errors increase – the effect of the observation is to make the error of the forecast started from x a greater than the error of the forecast started from x b

9 Beneficial impacts : - associated with observations more accurate than the background in regions where adjoint sensitivity gradients are large - extreme beneficial impacts from isolated observations indicate the need for greater observation density Non-beneficial impacts: - not expected, all observations should decrease forecast error - if occurs, look for problems in data QC, instrument accuracy, model error, specification of assimilation error statistics Identify observation impacts - Best Outcome: - many observations that produce equal or similar impacts, not few, isolated observations that produce large impacts

10 Three science issues need to be considered: 1.Observation data impacts are relevant only for the selected forecast error cost function. 2. Observation data impacts depend strongly on the assimilation method and the forecast model. 3.Accuracy of observation data impact equation subject to same limitations that apply to the adjoint of the forecast model – simplified physics, TLM assumptions Science issues in data impact calculations -

11 cost function can be any differentiable scalar measure of forecast model accuracy: single model variable (temperature, salinity, velocity) derived variable (sound speed) or some combination (integral energy norm as in NWP systems) impact of observations on all aspects of the forecast may not be directly estimated unless multiple cost functions are used. this project will develop and evaluate multiple ocean forecast error cost functions of Navy interest, for example: errors in position and strength of oceanographic features (fronts and eddies) acoustic prediction errors derived from forecast ocean state Issue 1: Data Impacts Relevant only for Selected Forecast Error Cost Function

12 quality of the forecast, assimilation statistics, and other issues with different models and assimilation systems may lead to quantitative differences in estimates of observation impact. in this project, data impact systems will be developed for: multiple ocean models (HYCOM and NCOM), and multiple data assimilation procedures (3DVar and 4DVar) this will allow determination of robustness of impact results when model domains and assimilated observations overlap: global HYCOM vs. regional NCOM both systems use same ocean data quality control system Issue 2: Impacts Depend on Assimilation Method and Forecast Model

13 tangent linear assumptions and approximated physics affect accuracy of forecast model adjoint. observation innovation vector metric can be used as a check: Issue 3: Accuracy of Data Impacts Subject to Forecast Model Adjoint Limitations is an approximation of the full non-linear model error (e f - e g ) Metric can be used as a check on accuracy of tangent linear assumptions in forecast model adjoint. Metric can be used to determine forecast lengths over which tangent linear assumptions are valid. Langland and Baker (2004) NOGAPS NOGAPS adjoint errors ~75% full model error

14 by routine monitoring and assessment of the value of observations assimilated in HYCOM and NCOM. by immediate feedback on the impact of new observing systems deployed in support of Navy exercises. by providing locations (adjoint sensitivity gradients) where forecast errors are sensitive to the initial conditions. by providing guidance on where additional observations, or better use of existing data (data selection, data thinning, quality control), may have the greatest potential to reduce forecast error. Payoff: Project will address major challenges in ocean observing and prediction

15 Applications: Multiple Uses of Adjoint-based System ApplicationInnovationsCost FunctionPurpose Observation Impact* Real observations (all known) Forecast error (known) Evaluate impacts of observations on forecast error Hypothetical Observations Real and simulated observations (some known, some unknown) Forecast error or model variable (known) Develop more optimal configurations of observing systems Targeted Observing Simulated observations (all unknown) Proxy for forecast error (unknown) Impact of adding new observations at some future time Hypothetical Observations and Targeted Observing are variants of core “Observation Impact” system: Langland and Baker (2004) NRL/MR/ *focus of this new start proposal

16 3DVar and 4DVar Developed directly from analysis codes NCOM – developed at NRL SSC in 6.2 project HYCOM – developed at French SHOM* Data Assimilation Adjoint Forecast Model Adjoint Develop Forecast Error Cost Functions Summary: System Components *Service Hydrographique Oceanographique de la Maritime System Integration: NCOM with 3DVar and 4DVar HYCOM with 3DVar System Evaluation: Navy Exercise Areas Ocean Sampling Experiments Global HYCOM Forecasts

17 END

18 COAMPS forecast error adjoint sensitivities to NCODA SST lower boundary conditions – sensitivities evolve over time with changing weather patterns 24 hrs Adjoint sensitivity gradients indicate where observations are likely to impact forecast error Kuroshio Convective Complexes Adjoint sensitivity maps - From: Amerault and Doyle (2008)

19 NOGAPS: Radiosonde Profile Observation Impact 1 Jan – 28 Feb 2006 Most beneficial (< J kg -1 ) Beneficial (-0.01 to -0.1 J kg -1 ) Non-beneficial (0.01 to 0.1 J kg -1 ) Most beneficial (04320) Least beneficial (84401) Many moderately beneficial Radiosonde impacts in CONUS and Europe – “best outcome” criteria Langland and Baker (2004)

20 Argo: Oceanographic Analog to Radiosonde Network Floats profile from ~2000 m to the surface every 10 days, measuring temperature and salinity as a function of depth. Adjoint-based system will determine impact of T,S data from each float cycle on reducing HYCOM and NCOM forecast errors.