Jeffrey Anderson, NCAR Data Assimilation Research Section

Slides:

Advertisements

Similar presentations

30 th September 2010 Bannister & Migliorini Slide 1 of 9 High-resolution assimilation and weather forecasting Ross Bannister and Stefano Migliorini (NCEO,

Advertisements

Motivation The Carolinas has had a tremendous residential and commercial investment in coastal areas during the past 10 years. However rapid development.

Effects of model error on ensemble forecast using the EnKF Hiroshi Koyama 1 and Masahiro Watanabe 2 1 : Center for Climate System Research, University.

1 アンサンブルカルマンフィルターによる大気海洋結合モデルへのデータ同化 On-line estimation of observation error covariance for ensemble-based filters Genta Ueno The Institute of Statistical.

3.11 Adaptive Data Assimilation to Include Spatially Variable Observation Error Statistics Rod Frehlich University of Colorado, Boulder and RAL/NCAR Funded.

Empirical Localization of Observation Impact in Ensemble Filters Jeff Anderson IMAGe/DAReS Thanks to Lili Lei, Tim Hoar, Kevin Raeder, Nancy Collins, Glen.

Initializing a Hurricane Vortex with an EnKF Yongsheng Chen Chris Snyder MMM / NCAR.

ASSIMILATION of RADAR DATA at CONVECTIVE SCALES with the EnKF: PERFECT-MODEL EXPERIMENTS USING WRF / DART Altuğ Aksoy National Center for Atmospheric Research.

Advanced data assimilation methods- EKF and EnKF Hong Li and Eugenia Kalnay University of Maryland July 2006.

Accounting for correlated AMSU-A observation errors in an ensemble Kalman filter 18 August 2014 The World Weather Open Science Conference, Montreal, Canada.

Ensemble Data Assimilation and Uncertainty Quantification Jeffrey Anderson, Alicia Karspeck, Tim Hoar, Nancy Collins, Kevin Raeder, Steve Yeager National.

ESA DA Projects Progress Meeting 2University of Reading Advanced Data Assimilation Methods WP2.1 Perform (ensemble) experiments to quantify model errors.

EnKF Overview and Theory

Observing Strategy and Observation Targeting for Tropical Cyclones Using Ensemble-Based Sensitivity Analysis and Data Assimilation Chen, Deng-Shun 3 Dec,

Eidgenössisches Departement des Innern EDI Bundesamt für Meteorologie und Klimatologie MeteoSchweiz Statistical Characteristics of High- Resolution COSMO.

The fear of the LORD is the beginning of wisdom 陳登舜 ATM NCU Group Meeting REFERENCE : Liu., H., J. Anderson, and Y.-H. Kuo, 2012: Improved analyses.

Computational Issues: An EnKF Perspective Jeff Whitaker NOAA Earth System Research Lab ENIAC 1948“Roadrunner” 2008.

Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss High-resolution data assimilation in COSMO: Status and.

MPO 674 Lecture 20 3/26/15. 3d-Var vs 4d-Var.

STEPS: An empirical treatment of forecast uncertainty Alan Seed BMRC Weather Forecasting Group.

11 Background Error Daryl T. Kleist* National Monsoon Mission Scoping Workshop IITM, Pune, India April 2011.

Pg 1 A Loosely Coupled Ocean-Atmosphere Ensemble Assimilation System. Tim Hoar, Nancy Collins, Kevin Raeder, Jeffrey Anderson, NCAR Institute for Math.

Data assimilation, short-term forecast, and forecasting error

Data assimilation and forecasting the weather (!) Eugenia Kalnay and many friends University of Maryland.

Retrieval of Moisture from GPS Slant-path Water Vapor Observations using 3DVAR and its Impact on the Prediction of Convective Initiation and Precipitation.

Deutscher Wetterdienst Vertical localization issues in LETKF Breogan Gomez, Andreas Rhodin, Hendrik Reich.

Application of COSMIC refractivity in Improving Tropical Analyses and Forecasts H. Liu, J. Anderson, B. Kuo, C. Snyder, and Y. Chen NCAR IMAGe/COSMIC/MMM.

Local Predictability of the Performance of an Ensemble Forecast System Liz Satterfield and Istvan Szunyogh Texas A&M University, College Station, TX Third.

Preliminary results from assimilation of GPS radio occultation data in WRF using an ensemble filter H. Liu, J. Anderson, B. Kuo, C. Snyder, A. Caya IMAGe.

A Random Subgrouping Scheme for Ensemble Kalman Filters Yun Liu Dept. of Atmospheric and Oceanic Science, University of Maryland Atmospheric and oceanic.

Mesoscale Atmospheric Predictability Martin Ehrendorfer Institut für Meteorologie und Geophysik Universität Innsbruck Presentation at 14th ALADIN Workshop.

Pg 1 SAMSI UQ Workshop; Sep Ensemble Data Assimilation and Uncertainty Quantification Jeff Anderson National Center for Atmospheric Research.

Future Directions in Ensemble DA for Hurricane Prediction Applications Jeff Anderson: NCAR Ryan Torn: SUNY Albany Thanks to Chris Snyder, Pavel Sakov The.

The application of ensemble Kalman filter in adaptive observation and information content estimation studies Junjie Liu and Eugenia Kalnay July 13th, 2007.

Korea Institute of Atmospheric Prediction Systems (KIAPS) ( 재 ) 한국형수치예보모델개발사업단 LETKF 앙상블 자료동화 시스템 테스트베드 구축 및 활용방안 Implementation and application of LETKF.

Mesoscale Assimilation of Rain-Affected Observations Clark Amerault National Research Council Postdoctoral Associate - Naval Research Laboratory, Monterey,

CMAQ PM 2.5 Forecasts Adjusted to Errors in Model Wind Fields Eun-Su Yang 1, Sundar A. Christopher 2 1 Earth System Science Center, UAHuntsville 2 Department.

ECMWF/EUMETSAT NWP-SAF Satellite data assimilation Training Course Mar 2016.

Kerry Emanuel Lorenz Center Massachusetts Institute of Technology

Alexander Loew1, Mike Schwank2

Shu-Chih Yang1,Kuan-Jen Lin1, Takemasa Miyoshi2 and Eugenia Kalnay2

Data Assimilation Research Testbed Tutorial

Predictability: How can we predict the climate decades into the future when we can’t even predict the weather for next week? Predictability of the first.

Predictability of Tropical Cyclone Intensity

Methods for dealing with spurious covariances arising from small samples in ensemble data assimilation Jeff Whitaker NOAA Earth.

Jeffrey Anderson, NCAR Data Assimilation Research Section

Jeff Anderson, NCAR Data Assimilation Research Section

Parallel Implementations of Ensemble Kalman Filters for Huge Geophysical Models Jeffrey Anderson, Helen Kershaw, Jonathan Hendricks, Nancy Collins, Ye.

Jeff Anderson, NCAR Data Assimilation Research Section

background error covariance matrices Rescaled EnKF Optimization

David Salstein, Edward Lorenz, Alan Robock, and John Roads

Jeff Anderson, NCAR Data Assimilation Research Section

Hui Liu, Jeff Anderson, and Bill Kuo

Ensemble variance loss in transport models:

UPDATE ON SATELLITE-DERIVED amv RESEARCH AND DEVELOPMENTS

Vertical localization issues in LETKF

New Approaches to Data Assimilation

Lidia Cucurull, NCEP/JCSDA

S. Kotsuki, K. Kurosawa, T. Miyoshi

Initial trials of 4DEnVAR

Assimilation of Global Positioning System Radio Occultation Observations Using an Ensemble Filter in Atmospheric Prediction Models Hui Liu, Jefferey Anderson,

Comparison of different combinations of ensemble-based and variational data assimilation approaches for deterministic NWP Mark Buehner Data Assimilation.

Impact of Assimilating AMSU-A Radiances on forecasts of 2008 Atlantic TCs Initialized with a limited-area EnKF Zhiquan Liu, Craig Schwartz, Chris Snyder,

Principles of the Global Positioning System Lecture 11

Data Assimilation Initiative, NCAR

Project Team: Mark Buehner Cecilien Charette Bin He Peter Houtekamer

Sarah Dance DARC/University of Reading

Fabien Carminati, Stefano Migliorini, & Bruce Ingleby

Data Assimilation of TEMPO NO2: Winds, Emissions and PBL mixing

Presentation transcript:

Jeffrey Anderson, NCAR Data Assimilation Research Section Ensemble Data Assimilation for Observations with Spatially and Temporally Correlated Errors Jeffrey Anderson, NCAR Data Assimilation Research Section CAHMDA VII, 21 Aug. 2017

Outline Dealing with correlated observation error in ensemble filters. Idealized correlated error. Difference observations. Explictly modeling instrument error. Comparing the two methods. Conclusions and recommendations. CAHMDA VII, 21 Aug. 2017

Most Observations Have Correlated Obs. Errors Examples: Satellite radiances: instrument bias and aging. In situ soil moisture: instrument plus siting representativeness. Rainfall: gauge deficiencies plus siting. CAHMDA VII, 21 Aug. 2017

Observation Error Time Series Example: Correlated Error AR1 with Variance 1. Single Step Cov 0.999. Fixed for all cases. CAHMDA VII, 21 Aug. 2017

Observation Error Time Series Example: Correlated Error AR1 with Variance 1. Single Step Cov 0.999. Fixed for all cases. Vary uncorrelated error variance, 0.01 CAHMDA VII, 21 Aug. 2017

Observation Error Time Series Example: Correlated Error AR1 with Variance 1. Single Step Cov 0.999. Fixed for all cases. Vary uncorrelated error variance, 0.1 CAHMDA VII, 21 Aug. 2017

Observation Error Time Series Example: Correlated Error AR1 with Variance 1. Single Step Cov 0.999. Fixed for all cases. Vary uncorrelated error variance, 1.0 CAHMDA VII, 21 Aug. 2017

Observation Error Time Series Example: Correlated Error AR1 with Variance 1. Single Step Cov 0.999. Fixed for all cases. Vary uncorrelated error variance, 10.0 CAHMDA VII, 21 Aug. 2017

Possible approaches to dealing with correlated obs error Ignore it (common), Add parameters to forward operator, estimate them, Model it explicitly (various ways), Time difference observations. CAHMDA VII, 21 Aug. 2017

1D Linear Exponential Growth Model True trajectory is always 0. Evolution is Perturbations grow exponentially in time. CAHMDA VII, 21 Aug. 2017

Outline Dealing with correlated observation error in ensemble filters. Idealized correlated error. Difference observations. Explictly modeling instrument error. Comparing the two methods. Conclusions and recommendations. CAHMDA VII, 21 Aug. 2017

Assimilating Correlated Observations CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results Exact Smoother Result. Can’t do better than this. CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results EAKF Poor Unless Uncorrelated Error Dominates RMSE Spread CAHMDA VII, 21 Aug. 2017

Two Types of Difference Observations CAHMDA VII, 21 Aug. 2017

Unlinked Difference Observations CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results Exact Unlinked Difference Obs Much worse. Exact Unlinked CAHMDA VII, 21 Aug. 2017

Uninked Difference Observations Unlinked Diff 1 Unlinked Diff 3 Unlinked Diff 5 Obs1 Obs2 Obs3 Obs4 Obs5 Obs6 CAHMDA VII, 21 Aug. 2017

Uninked Difference Observations Unlinked Diff 1 Unlinked Diff 3 Unlinked Diff 5 Obs1 Obs2 Obs3 Obs4 Obs5 Obs6 CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results EAKF is nearly exact for Unlinked Difference Obs. RMSE and Spread CAHMDA VII, 21 Aug. 2017

Linked Difference Observations CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results Exact linked Difference Obs Nearly Identical to Analytic. Linked CAHMDA VII, 21 Aug. 2017

Linked Difference Observations CAHMDA VII, 21 Aug. 2017

Linked Difference Observations CAHMDA VII, 21 Aug. 2017

Linked Difference Observations CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results EAKF Linked Diff. Obs. Good when correlated error dominates. RMSE Spread CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results Comparison to Just Using Raw Observations Obs CAHMDA VII, 21 Aug. 2017

Lorenz 63 Model Observing System 1 3 Instruments. Each has own correlated error. CAHMDA VII, 21 Aug. 2017

Lorenz 63 Model Observing System 2 1 Instrument measures x,y,z each time. CAHMDA VII, 21 Aug. 2017

L63 Results, Linked Difference Obs 3 Instruments 5 ensemble members. Adaptive inflation. Observations every 6 model timesteps. CAHMDA VII, 21 Aug. 2017

L63 Results, Linked Difference Obs 3 Instruments 1 Instrument 5 ensemble members. Adaptive inflation. Observations every 6 model timesteps. CAHMDA VII, 21 Aug. 2017

L63 Summary Difference obs better unless uncorrelated error variance dominates. Improvement greater for single instrument. Ensembles often under-dispersive (what a surprise!). CAHMDA VII, 21 Aug. 2017

Lorenz 96 Model, 40-variables Observing System 1 40 Instruments. Each has own correlated error. CAHMDA VII, 21 Aug. 2017

Lorenz 96 Model, 40-variables Observing System 2 1 instrument measures all 40 variables each time. CAHMDA VII, 21 Aug. 2017

L96 Results, Linked Difference Obs 40 Instruments 10 ensemble members. Adaptive inflation, 0.2 halfwidth localization. Observations every model timestep. CAHMDA VII, 21 Aug. 2017

L96 Results, Linked Difference Obs 40 Instruments 1 Instrument 10 ensemble members. Adaptive inflation, 0.2 halfwidth localization. Observations every model timestep. CAHMDA VII, 21 Aug. 2017

L96 Results, Linked Difference Obs Difference obs better unless uncorrelated error variance dominates. Improvement much greater for single instrument. Ensembles often over-dispersive. Dealing with time correlation harder than space correlation. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Evolution of surface pressure field every 12 hours. Has baroclinic instability: storms move east in midlatitudes. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Evolution of surface pressure field every 12 hours. Has baroclinic instability: storms move east in midlatitudes. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Evolution of surface pressure field every 12 hours. Has baroclinic instability: storms move east in midlatitudes. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Evolution of surface pressure field every 12 hours. Has baroclinic instability: storms move east in midlatitudes. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Evolution of surface pressure field every 12 hours. Has baroclinic instability: storms move east in midlatitudes. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Evolution of surface pressure field every 12 hours. Has baroclinic instability: storms move east in midlatitudes. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Evolution of surface pressure field every 12 hours. Has baroclinic instability: storms move east in midlatitudes. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Evolution of surface pressure field every 12 hours. Has baroclinic instability: storms move east in midlatitudes. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Evolution of surface pressure field every 12 hours. Has baroclinic instability: storms move east in midlatitudes. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Evolution of surface pressure field every 12 hours. Has baroclinic instability: storms move east in midlatitudes. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core: Grid Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles 30x60 horizontal grid, 5 levels. Surface pressure, temperature, wind components. 28,800 variables. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core: Observations Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Assimilate once per day. 0.2 radian localization. Observe each surface pressure grid point. Uncorrelated obs error variance 100 Pa. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core: Observations Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Uncorrelated obs error variance 100 Pa. Correlated obs error along ‘simulated polar orbiter track’. Vary ratio of correlated to uncorrelated obs error variance. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core: PS Results Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Linked difference better for large correlated error. Standard better for small correlated error. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core: T Results Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Linked difference better for large correlated error. Standard better for small correlated error. CAHMDA VII, 21 Aug. 2017

PS RMSE Structure: Large Uncorrelated Error, Ratio 4 Base errors largest in storm tracks. Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Linked difference errors largest in broad tropical band. CAHMDA VII, 21 Aug. 2017

PS RMSE Structure: Moderate Uncorrelated Error, Ratio 1 Base errors largest in storm tracks. Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Linked difference errors largest in broad tropical band. CAHMDA VII, 21 Aug. 2017

PS RMSE Structure: Small Uncorrelated Error, Ratio 1/4 Base errors largest in storm tracks. Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Linked difference errors largest in broad tropical band. CAHMDA VII, 21 Aug. 2017

T RMSE Structure: Small Uncorrelated Error, Ratio 1/4 Base errors largest in tropics. Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles Linked difference errors have similar pattern. CAHMDA VII, 21 Aug. 2017

Low-Order Dry Dynamical Core Summary Linked difference obs better for large correlated error. Linked difference not sensitive to correlated error size. Adaptive inflation struggles with large correlated error. Could use base approach for uncorrelated obs, difference for correlated error obs. For example, base for sondes, difference for radiances. Difference obs allows assimilating before knowing correlated error characteristics. Slide 7: GPS occultation forward operators Non-local refractivity : (Sokolovskiy et al., MWR 133, 2200-2212) NEED DETAILS OF ALGORITHM step size??? upper limits grid pictures for different latitudes Three pictures exist: ~jla/talks_and_meetings/2009/ams/cam_gps_talk/gps_lat* Expect minor differences with coarse grid, larger near poles CAHMDA VII, 21 Aug. 2017

Outline Dealing with correlated observation error in ensemble filters. Idealized correlated error. Difference observations. Explictly modeling instrument error. Comparing the two methods. Conclusions and recommendations. CAHMDA VII, 21 Aug. 2017

Modeling Correlated Observation Error Error in examples is AR1: (other types may need other methods). Given correlated error now, can predict it at later time. Have ensemble of model state. Also ensemble of correlated error for each instrument. CAHMDA VII, 21 Aug. 2017

Modeling Correlated Observation Error 1. Forecast: Advance model & correlated error ensembles. 2. Forward operator (for each ensemble member): Apply standard forward operator to state, H(x), Add correlated error. 3. Observation Increments: Compute normally. 4. State variable update: Use regression (ensemble Kalman gain) to update: Model state variables, Correlated observation variables. CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results 320 Member deterministic ensemble filter (EAKF) State CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results 320 Member deterministic ensemble filter (EAKF) State All results for 5000 steps after 1000 step spin-up. CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results 320 Member deterministic ensemble filter (EAKF) State 10 trials shown by *. Trial mean is line. All results for 5000 steps after 1000 step spin-up. CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results 320 Member deterministic ensemble filter (EAKF) State Exact asymptotic solution can be computed. Indistinguishable from 320 member ensemble. CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results 320 Member deterministic ensemble filter (EAKF) Obs. Error Exact asymptotic solution can be computed. Indistinguishable from 320 member ensemble. CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results Fails for small ensembles with large correlated error. 320 Member EAKF 20 Member EAKF CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results Fails for small ensembles with large correlated error. 10 Member EAKF 20 Member EAKF CAHMDA VII, 21 Aug. 2017

Ensemble Filters Scale Poorly for Random Fields Ensemble size > 1 is exact with no correlated obs error. Random walk evolution of correlated error is a problem. Can reduce this by reducing ‘randomness’ of ensemble. AR1 series for observation error is: Given a posterior ensemble estimate of e at previous time: Expected prior mean at next time is: Expected prior variance is: ‘Deterministic’ forecast for observation error: ‘Adjust’ ensemble to have exactly these statistics. CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results Deterministic works with smaller ensembles. Used hereafter. Nondeterm 20 Member Determ 10 Member Nondeterm 10 Member Determ 5 Member CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results Try multiplicative inflation of state. 10 Member inflated Optimal inflation gets large. Multiplicative inflation for obs error ensemble is bad. CAHMDA VII, 21 Aug. 2017

1D Exponential Growth Model Results Multiplicative inflation for state improves performance. 10 Member 10 Member inflated CAHMDA VII, 21 Aug. 2017

Lorenz 96 Model, 40-variables Observing System 1: 40 Instruments. Each has own correlated error. CAHMDA VII, 21 Aug. 2017

Lorenz 96 Model, 40-instruments 20 member EAKF. Optimal inflation. Localization halfwidth 0.2 Modeling obs error helps. Spread is deficient. CAHMDA VII, 21 Aug. 2017

Lorenz 96 Model, 40-variables Observing System 2: 1 instrument measures all 40 variables each time. CAHMDA VII, 21 Aug. 2017

Lorenz 96 Model, 1-instrument 20 member EAKF. Optimal inflation. Localization halfwidth 0.2 Modeling obs error helps. Spread is better than many instrument case. CAHMDA VII, 21 Aug. 2017

Outline Dealing with correlated observation error in ensemble filters. Idealized correlated error. Difference observations. Explictly modeling instrument error. Comparing the two methods. Conclusions and recommendations. CAHMDA VII, 21 Aug. 2017

Lorenz 96 Model, 40-instruments Time difference assimilation best for large correlated error. Terrible for small correlated error. 20 member EAKF. Optimal inflation. Localization halfwidth 0.2 CAHMDA VII, 21 Aug. 2017

Lorenz 96 Model, 1-instrument Time difference assimilation best for large correlated error. Not bad for small correlated error. 20 member EAKF. Optimal inflation. Localization halfwidth 0.2 CAHMDA VII, 21 Aug. 2017

Outline Dealing with correlated observation error in ensemble filters. Idealized correlated error. Difference observations. Explictly modeling instrument error. Comparing the two methods. Conclusions and recommendations. CAHMDA VII, 21 Aug. 2017

Conclusions Modeling correlated obs error ‘optimal’ for large ensemble. Sampling error is a problem for small ensembles. Multiplicative state inflation can reduce this problem. Additive inflation for obs error may help? Time difference obs effective for large correlated error. General things to keep in mind: Details of filtering problem determine best methods. Making models/filters more deterministic generally helps. CAHMDA VII, 21 Aug. 2017

Learn more about DART at: www.image.ucar.edu/DAReS/DART Anderson, J., Hoar, T., Raeder, K., Liu, H., Collins, N., Torn, R., Arellano, A., 2009: The Data Assimilation Research Testbed: A community facility. BAMS, 90, 1283—1296, doi: 10.1175/2009BAMS2618.1 CAHMDA VII, 21 Aug. 2017