Design and Validation of Observing System Simulation Experiments (OSSEs) Ronald Errico Goddard Earth Sciences Technology and Research Center at Morgan.

Slides:



Advertisements
Similar presentations
OSE meeting GODAE, Toulouse 4-5 June 2009 Interest of assimilating future Sea Surface Salinity measurements.
Advertisements

© The Aerospace Corporation 2014 Observation Impact on WRF Model Forecast Accuracy over Southwest Asia Michael D. McAtee Environmental Satellite Systems.
Design and Validation of the GMAO OSSE Ronald M. Errico Nikki Prive’ King-Sheng Tai Progress report presented to the GMAO 23 Feb
Quantitative retrievals of NO 2 from GOME Lara Gunn 1, Martyn Chipperfield 1, Richard Siddans 2 and Brian Kerridge 2 School of Earth and Environment Institute.
Transitioning unique NASA data and research technologies to the NWS 1 Radiance Assimilation Activities at SPoRT Will McCarty SPoRT SAC Wednesday June 13,
Improved Simulations of Clouds and Precipitation Using WRF-GSI Zhengqing Ye and Zhijin Li NASA-JPL/UCLA June, 2011.
Consideration of Dynamical Balances Ronald M. Errico Global Modeling and Assimilation Office, NASA Goddard Earth Sciences and Technology Center, UMBC A.
UNBIASED ESTIAMTION OF ANALYSIS AND FORECAST ERROR VARIANCES
Slide 1 Evaluation of observation impact and observation error covariance retuning Cristina Lupu, Carla Cardinali, Tony McNally ECMWF, Reading, UK WWOSC.
Towards Utilizing All-Sky Microwave Radiance Data in GEOS-5 Atmospheric Data Assimilation System Development of Observing System Simulation Experiments.
Ensemble-variational sea ice data assimilation Anna Shlyaeva, Mark Buehner, Alain Caya, Data Assimilation and Satellite Meteorology Research Jean-Francois.
Michiko Masutani NOAA/NWS/NCEP/EMC RSIS/Wyle Information Systems Introduction to OSSE and Summary of NCEP OSSEs
Observing Strategy and Observation Targeting for Tropical Cyclones Using Ensemble-Based Sensitivity Analysis and Data Assimilation Chen, Deng-Shun 3 Dec,
The Importance of Atmospheric Variability for Data Requirements, Data Assimilation, Forecast Errors, OSSEs and Verification Rod Frehlich and Robert Sharman.
ISDA 2014, Feb 24 – 28, Munich 1 Impact of ensemble perturbations provided by convective-scale ensemble data assimilation in the COSMO-DE model Florian.
1 ESTIMATING THE STATE OF LARGE SPATIOTEMPORALLY CHAOTIC SYSTEMS: WEATHER FORECASTING, ETC. Edward Ott University of Maryland Main Reference: E. OTT, B.
V Gorin and M Tsyrulnikov Can model error be perceived from routine observations?
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss High-resolution data assimilation in COSMO: Status and.
MPO 674 Lecture 20 3/26/15. 3d-Var vs 4d-Var.
Preliminary Results of Global Climate Simulations With a High- Resolution Atmospheric Model P. B. Duffy, B. Govindasamy, J. Milovich, K. Taylor, S. Thompson,
On Improving GFS Forecast Skills in the Southern Hemisphere: Ideas and Preliminary Results Fanglin Yang Andrew Collard, Russ Treadon, John Derber NCEP-EMC.
Data assimilation and observing systems strategies Pierre Gauthier Data Assimilation and Satellite Meteorology Division Meteorological Service of Canada.
The General Data Assimilation Problem Ronald Errico Goddard Earth Sciences Technology and Research Center at Morgan State University and Global Modeling.
Global Modeling and Assimilation Office Goddard Space Flight Center National Aeronautics and Space Administration The Simulation of Doppler Wind Lidar.
Ensemble Data Assimilation for the Mesoscale: A Progress Report Massimo Bonavita, Lucio Torrisi, Francesca Marcucci CNMCA National Meteorological Service.
Global Observing System Simulation Experiments (Global OSSEs) How It Works Nature Run 13-month uninterrupted forecast produces alternative atmosphere.
Modern Era Retrospective-analysis for Research and Applications: Introduction to NASA’s Modern Era Retrospective-analysis for Research and Applications:
1 JRA-55 the Japanese 55-year reanalysis project - status and plan - Climate Prediction Division Japan Meteorological Agency.
Data assimilation and forecasting the weather (!) Eugenia Kalnay and many friends University of Maryland.
Ensemble data assimilation in an operational context: the experience at the Italian Weather Service Massimo Bonavita and Lucio Torrisi CNMCA-UGM, Rome.
Weather forecasting by computer Michael Revell NIWA
Deutscher Wetterdienst Vertical localization issues in LETKF Breogan Gomez, Andreas Rhodin, Hendrik Reich.
Ronald M. Errico UMBC/GEST NASA/GMAO A plan for simulating current MW and IR radiance observations from the ECMWF Nature Run for the new “control” OSSE.
Page 1© Crown copyright 2004 SRNWP Lead Centre Report on Data Assimilation 2005 for EWGLAM/SRNWP Annual Meeting October 2005, Ljubljana, Slovenia.
Local Predictability of the Performance of an Ensemble Forecast System Liz Satterfield and Istvan Szunyogh Texas A&M University, College Station, TX Third.
LWG, Destin (Fl) 27/1/2009 Observation representativeness error ECMWF model spectra Application to ADM sampling mode and Joint-OSSE.
Implementation and Testing of 3DEnVAR and 4DEnVAR Algorithms within the ARPS Data Assimilation Framework Chengsi Liu, Ming Xue, and Rong Kong Center for.
I 5.11 Validation of the GMAO OSSE Prototype Runhua Yang 1,2 and Ronald Errico 1,3 1 Global Modeling and Assimilation office, GSFC, NASA 2 Science Systems.
The Infrastructure, Design and Applications of Observing System Simulation Experiments at NASA's Global Modeling and Assimilation Office By Ronald M. Errico.
The Impact of Data Assimilation on a Mesoscale Model of the New Zealand Region (NZLAM-VAR) P. Andrews, H. Oliver, M. Uddstrom, A. Korpela X. Zheng and.
Consideration of Representativeness Error In an OSSE Framework Before deciding specifically how observations are to be simulated, how instrument and representativeness.
2007/3/19 MRI, Tsukuba Recent developments of the local ensemble transform Kalman filter (LETKF) at JMA Takemasa Miyoshi (NPD/JMA) Collaborators: Shozo.
A step toward operational use of AMSR-E horizontal polarized radiance in JMA global data assimilation system Masahiro Kazumori Numerical Prediction Division.
Examination of Observation Impacts Derived from OSEs and Adjoint Models Ron Gelaro and Yanqiu Zhu NASA Global Modeling and Assimilation Office Ricardo.
École Doctorale des Sciences de l'Environnement d’Île-de-France Année Universitaire Modélisation Numérique de l’Écoulement Atmosphérique et Assimilation.
Infrared Sounding Data in the GMAO Data Assimilation System JCSDA Infrared Sounding Working Group (ISWG) 30 January 2009.
1 Satellite Winds Superobbing Howard Berger Mary Forsythe John Eyre Sean Healy Image Courtesy of UW - CIMSS Hurricane Opal October 1995.
Global vs mesoscale ATOVS assimilation at the Met Office Global Large obs error (4 K) NESDIS 1B radiances NOAA-15 & 16 HIRS and AMSU thinned to 154 km.
OSEs with HIRLAM and HARMONIE for EUCOS Nils Gustafsson, SMHI Sigurdur Thorsteinsson, IMO John de Vries, KNMI Roger Randriamampianina, met.no.
Radiance Simulation System for OSSE  Objectives  To evaluate the impact of observing system data under the context of numerical weather analysis and.
Assessing ACCESS-G Global Data Assimilation Using Adjoint-Based Forecast Sensitivity to Observations Jin LEE 1 and Paul GREGORY 2 Centre for Australian.
June 20, 2005Workshop on Chemical data assimilation and data needs Data Assimilation Methods Experience from operational meteorological assimilation John.
1 MODIS winds assimilation experiments and impact studies to date at the Met Office Howard Berger, Mary Forsythe, Met Office, Bracknell/Exeter, UK UW-CIMSS.
ECMWF/EUMETSAT NWP-SAF Satellite data assimilation Training Course Mar 2016.
Indirect impact of ozone assimilation using Gridpoint Statistical Interpolation (GSI) data assimilation system for regional applications Kathryn Newman1,2,
Forecast Sensitivity - Observation Impact (FSOI)
Data Assimilation Training
Ron Gelaro and Yanqiu Zhu NASA Global Modeling and Assimilation Office
Status of Generation of Simulated Observations for the Joint OSSE
Some General and Fundamental Requirements for Designing
Vertical localization issues in LETKF
5 July 2018 Nikki Privé R.M. Errico
FSOI adapted for used with 4D-EnVar
Use of ATOVS at DAO Joanna Joiner, Donald Frank, Arlindo da Silva,
ECMWF activities: Seasonal and sub-seasonal time scales
Comparison of different combinations of ensemble-based and variational data assimilation approaches for deterministic NWP Mark Buehner Data Assimilation.
How can an improved model yield higher RMS errors?
Results from the THORPEX Observation Impact Inter-comparison Project
Sarah Dance DARC/University of Reading
An OSSE Investigating a Constellation of 4- 5(
Presentation transcript:

Design and Validation of Observing System Simulation Experiments (OSSEs) Ronald Errico Goddard Earth Sciences Technology and Research Center at Morgan State University and Global Modeling and Assimilation Office at NASA Goddard Space Flight Center

Outline: 1.General Methodology 2.Simulation of observations and their errors 3.Validation of an OSSE: Assimilation metrics 4.Validation of an OSSE: Forecast metrics 5.Summary

Time Analysis Real Evolving Atmosphere, with imperfect observations. Truth unknown Climate simulation, with simulated imperfect “observations.” Truth known. Observing System Simulation Experiment Data Assimilation of Real Data

Applications of OSSEs 1.Estimate effects of proposed instruments (and their competing designs)on analysis skill by exploiting simulated environment. 2. Evaluate present and proposed techniques for data assimilation by exploiting known truth.

Requirements for an OSSE 1.A simulation of “truth”, generally produced as a free-running forecast produced by an NWP model (termed the “Nature Run”). 2.A simulation of a complete set of observations, drawn from truth. 3.A simulation of observational instrument plus representativeness errors to be added to the results of step 2. 4.A data assimilation system to ingest the results of step 3. All these steps must be done well enough to produce believable and useful metrics from step 4.

Validation of OSSEs As for any simulation, OSSE results apply to the assimilation of real data only to the degree the OSSE for such an application validates well with regard to relevant metrics. OSSE validity is first determined by carefully comparing a variety of statistics that can be computed in both the real assimilation and OSSE contexts.

Data assimilation is a fundamentally statistical problem Generally, averaging over long periods is required to obtain statistical significance. Only limited information about DAS behavior can be obtained from case studies. As assimilation systems improve and the number of observations used increases, the effects of any small subset of observations should decrease, and thus even longer averaging periods should be required to achieve statistical significance.

Choice of a Nature Run 1.A good simulation of nature in all important aspects 2.Ideally, individual realizations of the NR should be indistinguishable from corresponding realizations of nature (e.g., analyses) at the same time of year. 3.Since a state-of-the-art OSSE will require a cycling DAS, the NR should have temporal consistency. 4.For either 4DVAR or FGAT 3DVAR, NR datasets should have high frequency (i.e., < 6 hours) 5.Since dynamic balance is an important aspect of the atmosphere affecting a DAS, the NR datasets should have realistic balances. 6. For these and other reasons, using a state-of-the-art NWP model having a demonstrated good climatology to produce NR data sets is arguably the best choice.

ECMWF Nature Run 1.Free-running “forecast” 2.Operational model from T511L91 reduced linear Gaussian grid (approx 35km) 4.10 May 2005 through 31 May hourly output 6.SST and sea ice cover is real analysis for that period GMAO Nature Run 1.A 2012 model 2.A 10 km global grid 3.Hourly output 4.Aerosol and trace gases included 5.Available in 2013

Simulation of Observations 1. Any observation that can be assimilated can be simulated! 2. Differences between the H used to assimilate and simulate will be interpreted by the DAS as a representativeness error 3. Therefore, as more realism is modeled for the observation simulation compared to the assimilation, more representativeness error is introduced, including gross errors that must be identified by the DAS quality control. 4.It may be necessary to compensate for deficiencies in the nature run (e.g., unrealistic cloud cover) when simulating observations.

Dependency of results on errors Consider Kalman filter applied to linear model Daley and Menard 1993 MWR

Characteristics of Real Observation Errors 1.Generally unknown 2.Even statistics not well known 3.Often biased 4.Correlated (maybe even with background error) 5.Include gross errors 6.Generally non-Gaussian (a result of 5 or some basic physics; e.g. nonlinearity)

Simulation of Observation Errors 1.When simulating observations, there is no instrument and thus no instrument error. It therefore needs to be explicitly simulated and added, unless it is negligible. 2. An error of representativeness is generally implicitly created: 3.The real implicit rep. error is likely underestimated by this implicit component. It needs to also be simulated and added. 4.If real observation errors are correlated, the simulated ones must also be if the OSSE is to verify.

NCEP/GMAO GSI (3DVAR) scheme Resolution: approx. ½ degree, 72 levels Evaluation for 1-31 July 2005, with 2 week, accelerated spin up in June. Observations include all “conventional” observations available in 2005 (except for precipitation rates) and all radiance available from MSU, AMSU-A/B, HIRS-2/3, and AIRS instruments. Tuning parameters for simulated observation errors are error standard deviations, fractions of error variances for correlated components, vertical and horizontal correlation length scales Further details in Q.J.R.Met. Soc., 2012 Errico et al. and Privé et al. Assimilation System

Locations of QC-accepted observations for AIRS channel 295 at 18 UTC 12 July Real Simulated

Standard deviations of QC-accepted y-H(xb) values (Real vs. OSSE) AIRS AQUA AMSU-A NOAA-15 RAOB U Wind RAOB q

Horizontal correlations of y-H(xb) AMSU-A NOAA-15 Chan 6 GLOB RAOB T 700 hPa NHET GOES-IR SATWND 300 hPa NHET GOES-IR SATWND 850 hPa NHET

Square roots of zonal means of temporal variances of analysis increments T OSSE T Real U OSSEU Real

Time mean Analysis increments T 850 hPa U 500 hPa OSSEReal

Mean anomaly correlations OSSE vs Real Data: Forecasts

U-Wind RMS error: July Solid lines: 24 hour RMS error vs analysis Dashed lines: 120 hr forecast RMS error vs analysis

July Adjoint: dry error energy norm

Fractional reduction of zonal means of temporal variances of analysis errors compared with background errors uT

Warnings Past problems with some OSSEs 1.Some OSSEs use a very reduced set of as a control 2.Some OSSEs have very limited validation 3.Some OSSEs are based on very limited “case studies” 4.Some OSSEs use unrealistic obs errors (e.g., no rep. error) 5.Some OSSEs use a deficient NR

Warnings General criticisms of OSSEs 1.In OSSEs, the NR and DAS models are generally too alike, therefore underestimating model error and yielding overly-optimistic results. 2.When future specific components of the observing systems are deployed, the system in general will be different as will the DAS techniques, and therefore the specific OSSE results will not apply. 3. OSSEs are just bad science!

Response to Warnings 1.Design OSSEs thoughtfully 2.Consider implications of all assumptions 3.Validate OSSEs carefully. 4.Specify reasonable observation error statistics. 5.Avoid conflicts of interest. 6.Avoid over-selling results. 7.Only attempt to answer appropriate questions 8.Be skeptical of others’ works 9.Be critical of your own work (This is science!)