Observing System Evaluation Task Team Peter Oke 1 and Gilles Larnicol 2 1 CSIRO – Centre for Australian Weather and Climate Research 2 Collecte Localisation Satellites

Talk Outline GOV OSE TT mission How the TT works … Membership proposal Relationship to other GOV TTs Relationships to broader community Past activities Future plans >>> action items from 2 nd GODAE OSE WS

OSE TT mission 1.Demonstrate the value of existing observing systems –support observational agencies / patrons by demonstrating the importance of each observation platform to GODAE 2.Contribute to the ongoing design and assessment of the GOOS –where are the developing gaps / problems –what additional observations would be beneficial … through coordinated community activities and capability building

How the OSE TT works? Sharing of experiences, expertise, etc, through workshops: –1 st GODAE-OOPC OSE/OSSE WS (Nov 2007) –2 nd GOV OSE WS (June 2009) Reactive activities – responding to requests: –e.g., demonstrating importance of Jason-1 continuation –evidence to support decisions in changes to GOOS components (e.g., Jason-1/2 switch to inter-leaved orbits … was an opportunity missed) Proactive activities – anticipating future requirements: –Building capability for more timely reactive activities (2 nd GODAE OSE WS) and coordinating community activities: Argo design (i.e., 3x3 degree global optimal; benefit of deep profiling; importance of marginal seas) design of altimeter missions/constellations –importance of high quality reference mission –delivery of Cryosat-2 ocean product –wide vs single swath benefits of SSS In practice … the co-chairs will coordinate, and facilitate OSE activities; and collate “evidence” and deliver to GOV co-chairs for engagement with patrons. … proposals to ET-OOFS for the statement of guidance.

Membership proposal Co-chairs Core members from observational agencies Core members from each GODAE group Invited experts Action ALL >>> please provide co-chairs with recommendations for core members Action co-chairs >>> formalise membership through invitation

Relationships to other GOV TTs Inter-comparison and validation TT Coastal and Shelf Seas TT OSE TT 1.Demonstrate the value of existing observing systems >>> partly achieved through routine demonstration of forecast & analysis skill, and how this changes when the observing system changes (e.g., altimeter safe-hold, reduced Argo capacity, etc) Capability building and activities Common metrics & diagnostics

Relationships to broader community OOPC, Clivar, … GOOS, GCOS, … OSTST, Argo, GHRSST, … should GOV OSE TT become a GOV-OOPC-ETOOFS - SCOR working group? … our intention is to be inclusive We recognise that: GODAE systems critically depend on the availability of ocean observations … but most observing systems are not maintained for GODAE activities – they are primarily maintained for climate monitoring. To maintain/acquire relevance and influence, GODAE must proactively contribute to GOOS planning/discussions. GODAE must contribute positively – and not contradict, or compete with our more climate-centric colleagues.

Past Activities

GODAE Symposium contributions Oke, P. R., M. Balmaseda, J. Cummings, E. Dombrowsky, Y. Fujii, S. Guinehut, G. Larnicol, P.-Y. Le Traon, M. Martin, 2008: Observing System Evaluation, Proceedings of the GODAE Final Symposium, Nice, France Oke, P. R., M. Balmaseda, M. Benkiran, J. A. Cummings, E. Dombrowsky, Y. Fujii, S. Guinehut, G. Larnicol, P.-Y. Le Traon, and M. J. Martin, 2009: Observing System Evaluations using GODAE systems, Oceanography, accepted. Ocean Obs 09 contributions White paper - Ocean Observing System Evaluation G. Larnicol, P. R. Oke, M. A. Balmaseda, M. Benkiran, J. A. Cummings, E. Dombrowsky, Y. Fujii, S. Guinehut, P.-Y. Le Traon, M. J. Martin … Oceanography paper + OceanObs09 White paper = GOAE symposium paper Poster - Ocean Observing System Evaluation: Towards Routine Monitoring under GODAE OceanView P. R. Oke, G. Larnicol … summaries the outcomes of 2 nd GODAE OSE WS

Conventional methods: OSEs Altimeter SST XBT Moorings ForecastorBGFAnalysisorForecast Altimeter SST T/S XBT Moorings Evaluation/ Validation Assimilated observations Assimilated and with-held observations Assimilate real observations Systematically with-hold observation types Conventional method for demonstrating benefit/value of existing observation platforms … but very expensive and not always easy to interpret T/S

Conventional methods: OSSEs ForecastorBGFAnalysisorForecast Evaluation/ Validation Assimilated “observations” Simulated observations Assimilate pretend “observations” … from a model Systematically include different observation types … including future observation types Conventional method for demonstrating the potential benefit/value of future/possible observation platforms

Number of altimeter missions: DUACS system Pascual A., Y. Faugère, G. Larnicol and P.-Y. Le Traon (2006) Improved description of the ocean mesoscale variability by combining four satellite altimeters, Geophys. Res. Lett., 33, doi: /2005 GL RMS of SLA signal using data from 4 missions Difference between RMS of SLA signal using data from 4 missions and 2 missions Approximately half the SLA signal is lost when data from only 2 missions is used. Errors of up to 10 cm. 1/3 o DUACS system October August 2003 during the T/P interleaved mission … other studies have demonstrated that 4 altimeters in NRT is approximately equivalent to 2 altimeters in DT

NO ALTIM NO Argo NO SST NO DATAALL DATA Complimentary data types: mesoscale prediction Oke, P. R., and A. Schiller (2007): Impact of Argo, SST and altimeter data on an eddy-resolving ocean reanalysis. Geophys. Res. Lett., 34, L19601, doi: /2007GL No SSTNo ALTIM 1/10 o Bluelink system 6-month long OSEs starting December 2005

Assessing the value of future observation types: SMOS Tranchant, B., C.E. Testut, L. Renault, N. Ferry, F. Birol, P. Brasseur (2008) Expected impact of the future SMOS and Acquarius ocean surface salinity missions in the Mercator Ocean operational systems: new perspectives to monitor ocean circulation. Remote Sens. Env., 112, SMOS/Aquarius data should be very beneficial … but depends on errors 1/3 o Mercator System: North Atlantic SEEK filter 1-year OSSEs 2003 SMOS errors are spatially ~ psu depending on brightness temp, SST & wind. SMOS w/ 2x expected errors SMOS w/ 1/2x expected errors SMOS w/ expected errors Control Run

Future Plans

2nd GODAE Observing System Evaluation Workshop: Towards Routine Monitoring 4-5 June 2009 CLS/Mercator-Ocean, Toulouse, France Attendees included representation from: -Bluelink, Mercator, UK Met, TOPAZ, JMA-MRI, NRL-HYCOM, CLS -JCOMM-OPS (Argo, in situ programs), NOAA (altim), CNES (altim) Presentations covered: -Various OSE & OSSE experiments (e.g., TAO vs Argo vs TAO+Argo; impact of SSS; short-range to seasonal applications and coastal applications) -Existing GODAE system metrics/diagnostics & evaluation activities -NWP experience (THORPEX) from special guest (Florence Rabier) -Proposed action items

Workshop Objectives Key question: How can we use GODAE systems for routine monitoring of the GOOS? Key objective: Establish a plan for coordinated routine monitoring of the GOOS using GODAE systems – - what will we do? - how will we do it? - when will we do it? how regularly?

Workshop Objectives Scope: –physical, biological, ocean-atmosphere ? –global, regional, coastal ? –what is required for each observing system from GODAE ? Appropriate experiments: –Conventional OSEs/OSSEs –analysis / forecast sensitivity –assimilation diagnostics (obs-space) –other … Method of delivery: –Central host vs distributed web pages –ET-OOFS – JCOMM – WMO ?

Proposed Action Items 1.Short-term actions re: Jason-1 continuation 2.Facilitating community OSSEs 3.Community analysis of assimilation diagnostics 4.Capability building for routine analysis sensitivity

1. Short-term actions re: Jason-1 continuation Motivation: We need to provide evidence of the need to continue Jasn-1 operations … required by July 2009 Actions: Jim Cummings send demonstration of benefits of Jason- 1 for wave height forecasts Gary Brassington and Daniel Lea send demonstrations of the benefits of Jason-1 for GODAE forecasts … send to PO & GL (we’ll collate and forward to AS & ED) Deadline – 15 July 2009

2. Facilitating community OSSEs Motivation: timely response to potential developments in possible new observing system platforms We agreed to: –generate/identify a single nature run to be used for OSSE activities –agree to collate a list of key questions that we need to ask –communicate what OSSEs we can find resources for (e.g., through students, PostDocs etc)

2. Facilitating community OSSEs Specific immediate action items ALL WS participants >>> Send ideas for possible nature run to PO & GL ALL WS participants >>> Send key OSSE questions to PO & GL PO & GL >>> seek input on OSSE questions from obs agencies and facilitate access to community nature run –e.g., Do we need a reference altim mission? Design of Jason-4 Cryosat 2 Drifting mission with addition error When is obs error for altim, sst, sss etc too large to be useful? Is a 3x3 Argo array optimal? What are the benefits of SSS? Impact of Argo in EEZs Deadline for input: 15 July 2009

3. Community analysis of assimilation diagnostics Motivation: facilitate timely demonstration of the impact of “observing system events” (e.g., safe hold of altimeter mission; addition of new observations; …) Each group already calculates and archives common diagnostics in observation space: –for each observation type - SLA, SST, T/S(z): w o, Hw b, Hw a, including x, y, z, and t –WS participants agreed*** to make these fields available so that they can routinely be synthesised for different domains (e.g., each WBC region, each tropical basin, etc.) and disseminating via the GODAE website Each group agreed*** to provide analysis and 0-5 d forecasts of: –U, V, SLA, SST, and MLD on native grid (all 2d fields) … to quantify forecast errors and how these change with “events” … to enable qualitative animations with drifter trajectories overlaid … to estimate eddy statistics (paths, intensity, etc) … perhaps even just for the 1 st and 3 rd Monday of each month ***Some groups need permission >>> PO & GL will generate a metrics document, based on the metrics already established by the inter-comparisons/validation TT

4. Capability building for routine analysis sensitivity Motivation: routine evaluation of impact of each observation to feed back to observation agencies Analysis self-sensitivity is possible for all groups –requires an additional analysis, using perturbed observations –and an inexpensive calculation, in observation space to compute the trace, and sub-trace of different variables, regions etc. Each group agreed to develop this capability in research mode? Each group agreed to perform this analysis for a demonstration case … and work towards routine performance >>> PO to circulate relevant reverences and sample code

Method of delivery Bluelink Mercator NRL JMA TOPAZ UK Met Example page only

Points for comment Feedback of WS action items 1.Short-term actions re: Jason-1 continuation 2.Facilitating community OSSEs 3.Community analysis of assimilation diagnostics 4.Capability building for routine analysis sensitivity GOV OSE TT  GOV-OOPC-ETOOFS TT? –does the extra level of management/reporting return any practical benefit? –thoughts on proposition for joining/leading a SCOR OSE WG? Action items on GOV ST >>> please provide OSE TT co-chairs with recommendations for core members

Analysis self-sensitivities Step 1: Perform an analysis Step 2: perturb observations (d*) according to their expected error (R) Step 3: Perform another analysis Step 4: Compute self-sensitivities: d i is the original i th obs; d i * is the perturbed i th obs; Ha i is the original analysis, interpolated to the obs; Ha i * is the new analysis, using d*, interpolated to the obs; HK ii is an estimate of the self-sensitivity. HK ii = (d i *-d i ) (Ha i *-Ha i ) / R ii … repeat steps 2-4 to check robustness of results. All elements of HK can be analysed, or the partial trace of HK can be averaged for different regions, variables, etc.

Analysis self-sensitivities Preliminary estimates of the Information Content (IC; %), degrees of freedom of signal (DFS) and the number of assimilated super-observations (# Obs) for the Bluelink reanalysis system in the region o E, 60oS-equator, computed for 1 January The scale for the IC is to the left and the scale for the DFS and # Obs is to the right.

Analysis sensitivity of Argo T over top 200 m depth 5-different realisations of the self- sensitivity (HK ii ) Averaged over top 200 m depth Shows some dependence on the perturbations to observations

Analysis sensitivity of Argo T over top 200 m depth If HK ii – std err >0.5  High If HK ii – std err >0 & < 0.5  Med If HK ii – std err <0  Low (?) AverageStandard errorInfluence

Analysis sensitivity of Argo T over top 200 m depth

Analysis sensitivity of Argo S over top 200 m depth

If HK ii – std err >0.5  High If HK ii – std err >0 & < 0.5  Med If HK ii – std err <0  Low (?) AverageStandard errorInfluence

Analysis sensitivity of Argo S over top 200 m depth

Method of delivery Bluelink Mercator NRL JMA TOPAZ UK Met Example page only

… from Juliette and Gerald’s presentations: A key issue in altimetry is that missions with degraded quality are inevitable. It is expensive to maintain missions, so the operating agencies need to know when a degraded mission becomes useless. Q/ At what point is an altimeter not worth assimilating? When the errors get to what level? re: spatially correlated errors, not random errors. Conclusions that 3-4 altimeters are required are insufficient. We need to generate evidence that certain satellite constellations are needed for GODAE. Through routine monitoring we need to illustrate the impact of major events. –routine, online, NRT skill assessments Possibly consider annual OSSEs based on possible future missions: which option is better. We may need to have a OSSE experiments design ready for response. e.g., have a nature run completed, and a