3.11 Adaptive Data Assimilation to Include Spatially Variable Observation Error Statistics Rod Frehlich University of Colorado, Boulder and RAL/NCAR Funded.

Slides:



Advertisements
Similar presentations
Introduction to data assimilation in meteorology Pierre Brousseau, Ludovic Auger ATMO 08,Alghero, september 2008.
Advertisements

Computational Statistics. Basic ideas  Predict values that are hard to measure irl, by using co-variables (other properties from the same measurement.
The Simple Regression Model
1 High resolution SST products for 2001 Satellite SST products and coverage In situ observations, coverage Quality control procedures Satellite error statistics.
Environmental Data Analysis with MatLab Lecture 23: Hypothesis Testing continued; F-Tests.
Improving radar Doppler wind information extraction Yong Kheng Goh, Anthony Holt University of Essex, U. K. Günther Haase, Tomas Landelius SMHI, Sweden.
Chapter 6 Introduction to Sampling Distributions
ASSIMILATION of RADAR DATA at CONVECTIVE SCALES with the EnKF: PERFECT-MODEL EXPERIMENTS USING WRF / DART Altuğ Aksoy National Center for Atmospheric Research.
Fall 2006 – Fundamentals of Business Statistics 1 Chapter 6 Introduction to Sampling Distributions.
Regionalized Variables take on values according to spatial location. Given: Where: A “structural” coarse scale forcing or trend A random” Local spatial.
Statistics, data, and deterministic models NRCSE.
Brian Ancell, Cliff Mass, Gregory J. Hakim University of Washington
Intro to Statistics for the Behavioral Sciences PSYC 1900
Statistics, data, and deterministic models NRCSE.
A comparison of hybrid ensemble transform Kalman filter(ETKF)-3DVAR and ensemble square root filter (EnSRF) analysis schemes Xuguang Wang NOAA/ESRL/PSD,
Standard error of estimate & Confidence interval.
Statistical Methods For Engineers ChE 477 (UO Lab) Larry Baxter & Stan Harding Brigham Young University.
1 G Lect 11W Logistic Regression Review Maximum Likelihood Estimates Probit Regression and Example Model Fit G Multiple Regression Week 11.
Oceanography 569 Oceanographic Data Analysis Laboratory Kathie Kelly Applied Physics Laboratory 515 Ben Hall IR Bldg class web site: faculty.washington.edu/kellyapl/classes/ocean569_.
The Importance of Atmospheric Variability for Data Requirements, Data Assimilation, Forecast Errors, OSSEs and Verification Rod Frehlich and Robert Sharman.
Lidar Working Group on Space-Based Winds, Snowmass, Colorado, July 17-21, 2007 A study of range resolution effects on accuracy and precision of velocity.
Statistical Decision Making. Almost all problems in statistics can be formulated as a problem of making a decision. That is given some data observed from.
Stephan F.J. De Wekker S. Aulenbach, B. Sacks, D. Schimel, B. Stephens, National Center for Atmospheric Research, Boulder CO; T. Vukicevic,
Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss High-resolution data assimilation in COSMO: Status and.
Regional climate prediction comparisons via statistical upscaling and downscaling Peter Guttorp University of Washington Norwegian Computing Center
Dusanka Zupanski And Scott Denning Colorado State University Fort Collins, CO CMDL Workshop on Modeling and Data Analysis of Atmospheric CO.
Cooperative Research Programs (CoRP) Satellite Climate Studies Branch (SCSB) 1 1 Reconstruction of Near-Global Precipitation Variations Thomas Smith 1.
1 Motivation Motivation SST analysis products at NCDC SST analysis products at NCDC  Extended Reconstruction SST (ERSST) v.3b  Daily Optimum Interpolation.
11 Background Error Daryl T. Kleist* National Monsoon Mission Scoping Workshop IITM, Pune, India April 2011.
Rod Frehlich University of Colorado, Boulder, CO RAL/NCAR, Boulder, CO
Data assimilation, short-term forecast, and forecasting error
Spatial Analysis & Geostatistics Methods of Interpolation Linear interpolation using an equation to compute z at any point on a triangle.
Statistical Inference for the Mean Objectives: (Chapter 9, DeCoursey) -To understand the terms: Null Hypothesis, Rejection Region, and Type I and II errors.
6/29/2010 Wan-Shu Wu1 GSI Tutorial 2010 Background and Observation Error Estimation and Tuning.
1 Regression Analysis The contents in this chapter are from Chapters of the textbook. The cntry15.sav data will be used. The data collected 15 countries’
LWG, Destin (Fl) 27/1/2009 Observation representativeness error ECMWF model spectra Application to ADM sampling mode and Joint-OSSE.
Ocean Surface Current Observations in PWS Carter Ohlmann Institute for Computational Earth System Science, University of California, Santa Barbara, CA.
WCRP Extremes Workshop Sept 2010 Detecting human influence on extreme daily temperature at regional scales Photo: F. Zwiers (Long-tailed Jaeger)
Stochastic Hydrology Random Field Simulation Professor Ke-Sheng Cheng Department of Bioenvironmental Systems Engineering National Taiwan University.
ESTIMATION METHODS We know how to calculate confidence intervals for estimates of  and  2 Now, we need procedures to calculate  and  2, themselves.
Evidence in ISCCP for regional patterns of cloud response to climate change Joel Norris Scripps Institution of Oceanography ISCCP at 30 Workshop City College.
The application of ensemble Kalman filter in adaptive observation and information content estimation studies Junjie Liu and Eugenia Kalnay July 13th, 2007.
ENSEMBLE KALMAN FILTER IN THE PRESENCE OF MODEL ERRORS Hong Li Eugenia Kalnay.
Lec. 19 – Hypothesis Testing: The Null and Types of Error.
Statistical Inference for the Mean Objectives: (Chapter 8&9, DeCoursey) -To understand the terms variance and standard error of a sample mean, Null Hypothesis,
June 20, 2005Workshop on Chemical data assimilation and data needs Data Assimilation Methods Experience from operational meteorological assimilation John.
Early VALIDAR Case Study Results Rod Frehlich: RAL/NCAR Grady Koch: NASA Langley.
1 C.A.L. Bailer-Jones. Machine Learning. Data exploration and dimensionality reduction Machine learning, pattern recognition and statistical data modelling.
Estimating standard error using bootstrap
Chapter 6: Sampling Distributions
ESTIMATION METHODS We know how to calculate confidence intervals for estimates of  and 2 Now, we need procedures to calculate  and 2 , themselves.
Why Model? Make predictions or forecasts where we don’t have data.
Jeffrey Anderson, NCAR Data Assimilation Research Section
NRCSE 2. Covariances.
Chapter Six Normal Curves and Sampling Probability Distributions
New developments in the Graphical Turbulence Guidance (GTG) product
Basic Estimation Techniques
Statistical Methods For Engineers
OSSEs for Pacific Predictability Josh Hacker, NCAR
Stochastic Hydrology Random Field Simulation
Sampling Distribution
Sampling Distribution
Initialization of Numerical Forecast Models with Satellite data
Error statistics in data assimilation
CHAPTER 15 SUMMARY Chapter Specifics
Statistical tests for replicated experiments
ESTIMATION METHODS We know how to calculate confidence intervals for estimates of  and 2 Now, we need procedures to calculate  and 2 , themselves.
Forecast system development activities
Probabilistic Surrogate Models
Status of the Regional OSSE for Space-Based LIDAR Winds – Feb01
Presentation transcript:

3.11 Adaptive Data Assimilation to Include Spatially Variable Observation Error Statistics Rod Frehlich University of Colorado, Boulder and RAL/NCAR Funded by NSF (Lydia Gates)

Turbulence Small scale turbulence defines the observation sampling error or “error of representativeness” Critical component of the total observation error Turbulence and observation sampling error have large spatial variations Optimal data assimilation must include these variations

Spatial Spectra Robust description in troposphere Power law scaling Spectral level defines turbulence (ε 2/3 and C T 2 )

Structure Functions Alternate spatial statistic Also has power law scaling Structure functions (and spectra) from model output are filtered Corrections are possible by comparisons with in situ data Produce local estimates of turbulence defined by ε or C T 2 for adaptive data assimilation

RUC20 Analysis RUC20 model structure function In situ “truth” from GASP data Effective spatial filter (3.5Δ) determined by agreement with theory

DEFINITION OF TRUTH For forecast error, truth is defined by the spatial filter of the model numerics For the initial field (analysis) truth should have the same definition for consistency

Observation Sampling Error Truth is the average of the variable x over the LxL effective grid cell Total observation error Instrument error =  x Sampling error =  x The observation sampling pattern and the local turbulence determines the sampling error

Sampling Error for Velocity and Temperature in Troposphere Rawinsonde in center of square effective grid cell of length L

Sampling Error for Global Model Rawinsonde in center of grid cell Large variations in sampling error Dominant component of total observation error in many regions Most accurate observations in low turbulence regions

Optimal Data Assimilation Optimal assimilation requires estimation of total observation error covariance Requires calculation of instrument error which may depend on local turbulence (profiler, Doppler lidar) Requires calculation of sampling error Calculation of analysis error

Adaptive Data Assimilation Assume locally homogeneous turbulence around the analysis point r forecast (first-guess) N observations

Measurement Geometry Single observation at the analysis coordinate Multiple observations around the analysis coordinate Aircraft track

Analysis Error for u Velocity Instrument error is 0.5 m/s ( …... ) sampling ( ___ ) all rawin ( _ _ _ ) one rawin ( _. _.) lidar Turbulence is important > 5%

SUMMARY Turbulence produces large spatial variations in total observation error Optimal data assimilation using local estimates of turbulence reduces the analysis error

NCAR Collaborators Bob Sharman Francois Vandenberghe Yubao Liu Josh Hacker

Optimal Analysis Error Analysis error depends on forecast error and effective observation error forecast error effective observation error (local turbulence)

Analysis Error for Temperature Instrument error is 0.5 K (.... ) sampling error ( ___ ) all data ( ) single obs. ( ) aircraft Turbulence is important > 50%

UKMO 0.5 o Global Model Effective spatial filter (5Δ) larger than RUC The s 2 scaling implies only linear spatial variations of the field (smooth)

GFS 0.5 o Global Model Effective spatial filter (5Δ) is the same as UKMO

Estimates of Small Scale Turbulence Calculate structure functions locally over LxL square Determine best-fit to empirical model Estimate in situ turbulence level and ε ε 1/3

Climatology of Small Scale Turbulence Probability Density Function (PDF) of ε Good fit to the Log Normal model Parameters of Log Normal model depends on domain size L Consistent with large Reynolds number turbulence

Scaling Laws for Log Normal Parameters Power law scaling for the mean and standard deviation of log ε Consistent with high Reynolds number turbulence