Presentation is loading. Please wait.

Presentation is loading. Please wait.

DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 ARMY Research Lab and CIRA/CSU Collaboration on.

Published byAmice Hart Modified over 8 years ago

Similar presentations

Presentation on theme: "DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 ARMY Research Lab and CIRA/CSU Collaboration on."— Presentation transcript:

1 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 ARMY Research Lab and CIRA/CSU Collaboration on Assessing the value of ensemble based data assimilation/forecasting system in Army microscale meteorological applications Milija Zupanski CIRA Colorado State University (E-mail: ZupanskiM@CIRA.colostate.edu)

2 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 ARMY Research Lab and CIRA/CSU Collaboration on Assessing the value of ensemble based data assimilation/forecasting system in Army microscale meteorological applications White paper overview Proposed work: i - Structure and correction of model error (bias) at the microscale - understand the errors and try to correct them ii - Impact of mesoscale model forecast on initialization of microscale model - transfer of uncertainty information between scales iii -Optimal use of computer resources, including HPC, with new ensemble data assimilation methodology - HPC performance of ensemble data assimilation

3 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Extended Kalman Filter Equivalent to solving a quadratic minimization problem: Subject to P f - forecast error covariance P a - analysis error covariance R - observation error covariance K - nonlinear observation operator K - linearized observation operator (Jacobian) Direct solution:

4 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Maximum Likelihood Ensemble Filter (MLEF) Maximize posterior probability to obtain the conditional mode Equivalent to minimizing general non-quadratic cost function: No assumption regarding the (non)linearity of observation operator K ! Optimal state + uncertainty Embedded in optimization algorithm (2-3 minimization iterations!) Analysis perturbations only (no observation perturbations) Math library subroutines for matrix operations (BLAS, LAPACK) Mathematically consistent feed-back between data assimilation and ensemble forecasting

5 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Maximum Likelihood Ensemble Filter (MLEF) Forecast error covariance x – initial conditions r – model error (including bias) p i – i -th column of the square-root analysis erorr covariance matrix b i – i -th column of the square-root forecast erorr covariance matrix

6 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Change of variable (Hessian preconditioing) Maximum Likelihood Ensemble Filter (MLEF) Change of variable (Hessian preconditioing) Inversion via an eigenvalue decomposition of A:

7 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Maximum Likelihood Ensemble Filter (MLEF) Gradient calculation Need only forecast difference at observation location ! Use:

8 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Maximum Likelihood Ensemble Filter (MLEF) Analysis error covariance By-product of minimization algorithm L-BFGS: C-G:

9 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Ensemble Data Assimilation (EnsDA) Algorithm Background: a- NCEP operational algorithm structure - driven by UNIX scripts b- Weather Research and Forecasting (WRF) 3DVAR programming - definition of control, first-guess, as ‘type’ variables c- 4DVAR experience (NCEP Eta-model, CIRA/CSU RAMS model) - flow of data, user-friendly approach

10 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Ensemble Data Assimilation (EnsDA) Algorithm prep_ensda.sh cycle_ensda.sh WARM start: Copy files from previously completed cycle COLD start: Run randomly-perturbed ensemble forecasts to initialize fcst err cov icycle < N_cycles_max post.sh (post-processing) fcsterr_cov.sh - Prepare first-guess (background) vector - Prepare forecast error covariance (from ensembles) prep_obs.sh Given ‘OBSTYPE’ and ‘delobs’, select and copy available obs files assimilation.sh Iterative minimization of cost function, save current cycle output

11 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Ensemble Data Assimilation (EnsDA) Algorithm assimilation.sh script assimilation.sh iter < ioutmax forward.sh: Transformation from model space to observation space - Analysis error covariance calculation - Save current cycle output files Hessian preconditioning (only for iter=1) Gradient calculation (ensembles) Cost function calculation (diagnostic) Minimization (ensemble subspace) Step-length (line-search) Control variable update (transformation from ensemble subspace to model (physical) space)

12 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Ensemble Data Assimilation (EnsDA) Algorithm Organization chart../ensda/include: Include files (control variable list, first-guess list)../ensda/makefiles: Makefile_ensda, Makefile_’MODEL’../ensda/runscripts: Run EnsDA program (runensda, envir.sh)../ensda/scripts: Scripts used to run the EnsDA program../ensda/src: Source programs../’MODEL’/src: Model source programs (general)../’MODEL’/src_ensda: Model source programs (changes due to EnsDA)../exec/ensda: Executable directory../’Obsdir’: Observations directory../ensda/work: Working (temporary) directory../hold.’print’/current:General EnsDA output../hold.’print’/cycle1, cycle2,... :Cycle specific EnsDA output

13 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Ensemble Data Assimilation (EnsDA) Algorithm Options – envir.sh MODEL: WRF,UW-NMS,PU-NTU OBS: OBSTYPE, delobs,OBSFLAG STAT:Mode (max likelihood), Mean (ensemble mean) DA_TYPE:Filter, Smoother COVAR:Localized, Non-localized forecast error covariance N_CYCLES:Number of data assimilation cycles CYCLE_INTERVAL: Time interval between cycles ENS_SIZE:Number of ensemble members START:Cold (from scratch), Warm (from previous cycle) IOUTMAX:Number of minimization iterations MINIM_ALG:Minimization algorithm (C-G, L-BFGS) MPI_RUN:Parallel run (YES, NO) nCPU_ensfcst:No. of CPUs for ensemble fcst nCPU_da:No. of CPUs for data assimilation

14 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Ensemble Data Assimilation (EnsDA) Algorithm Control variables !--------------------------------------- max_num_of_cntrl_vrbls = 5 if(.not.allocated (cvar_list)) then allocate (cvar_list(1:max_num_of_cntrl_vrbls)) end if !--------------------------------------- ncv= 1 cvar_list(ncv)%ndim = 3 cvar_list(ncv)%start_index(1) = 1 cvar_list(ncv)%start_index(2) = 1 cvar_list(ncv)%start_index(3) = 1 cvar_list(ncv)%start_index(4) = 1 cvar_list(ncv)%end_index(1) = NNXP(1) cvar_list(ncv)%end_index(2) = NNYP(1) cvar_list(ncv)%end_index(3) = NNZP(1) cvar_list(ncv)%end_index(4) = 1 cvar_list(ncv)%name = 'T ' cvar_list(ncv)%stddev = 1.5 cvar_list(ncv)%description = 'Temperature ‘ !--------------------------------------- 6 p T F F 0 t T F T 1 q T F T 3 u T F F 0 v T F F 0 param1 F T F 0 param2 F T F 0 !-- first line number is the number of control variables defined --! !-- vrbl name, ic flag, param flag, bias flag, number of biases ----! !--- recall that vrbl name has 9 characters !!!!! !--- All inputs have to be separated by at least one blank space !!!../include/Cntrl_vrbl_list.h../namelists/Cntrl_vrbl_’MODEL’.h Optional control variable components: - initial conditions - model bias - empirical parameters

15 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Ensemble Data Assimilation (EnsDA) Algorithm Observations # Precip obs export iobs=02 #---------- specified by user --------- export OBSTYPE=precip export OBSFLAG=NOT let "delobs=1*60" ; export delobs #----------- end user ------------------ -------------------------- OBSTYPE – algorithm recognizable obs type delobs– time interval between two successive observation files OBSFLAG – Specifies if the OBSTYPE obs should be assimilated (YES, NOT) interface_’MODEL’_’OBSTYPE’.sh (model produced first-guess file => obs operators compatible first-guess file) runobs_’OBSTYPE’.sh (forward-only observation operator: first-guess => innovation vector R -1/2 [y-K(x)] )

16 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Ensemble Data Assimilation (EnsDA) Algorithm Model convert_vrbls_from_1d.sh run_’MODEL’.sh convert_vrbls_to_1d.sh - Data assimilation is defined using 1-d control variable - Model is defined using its own 1-d, 2-d, or 3-d, or 4-d variable - Need ‘convert’ as an interface between the 1-d and model specific variables../’MODEL’/src: (‘MODEL’_fcst.F) read namelists: ‘MODEL_DIMENSION’,’LENGTH’,’MODEL_CNSTS’ call: ‘read/write_model_ic’,’read/write_model_bias’,’read/write_model_param’../’MODEL’/src_ensda: ‘MODEL’_get_init_fcst.F – cold start only, get model_dimension file ‘read/write_model_ic.F’,’read/write_model_bias.F’,’read/write_model_param.F’

17 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Ensemble Data Assimilation (EnsDA) Algorithm Including model bias DO n=1,N_time !! Main time-loop. if(num_bias.gt.0) then nerr=N_time/max(1,num_bias) nbias=0 if((n.ne.N_time).and.(mod(n,nerr).eq.0.or.n.eq.1)) then nbias=nbias+1 call read_model_bias(...,u_bias,v_bias,t_bias,...,nbias,num_bias) endif. if(num_bias.gt.0) then u_phi(:,:,:)=aa*u_phi(:,:,:)+(1.-aa)*u_bias(:,:,:) v_phi(:,:,:)=aa*v_phi(:,:,:)+(1.-aa)*v_bias(:,:,:) t_phi(:,:,:)=aa*t_phi(:,:,:)+(1.-aa)*t_bias(:,:,:). u(:,:,:)=u(:,:,:)+u_phi(:,:,:) v(:,:,:)=v(:,:,:)+v_phi(:,:,:) t(:,:,:)=t(:,:,:)+t_phi(:,:,:) endif. END DO !! End main time loop

18 DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 Ensemble Data Assimilation (EnsDA) Algorithm MPI !-------------------------------------------------------------------- #ifdef MPI_USE include "mpif.h" #endif......... !==============start calculation=================== #ifdef MPI_USE CALL MPI_INIT(IERR) IF (IERR.NE. MPI_SUCCESS) STOP 'FAILED TO INIT MPI' CALL MPI_COMM_SIZE(MPI_COMM_WORLD, NPROC, IERR) CALL MPI_COMM_RANK(MPI_COMM_WORLD, MPIRANK, IERR) CALL MPI_BARRIER(MPI_COMM_WORLD,IERR) #else write(*,*) "This is a NO MPI run" MPIRANK=0 NPROC=1 #endif !================================================ !-- read ensemble size (NENS) …… !== define local dimensions and indexes allocate(jdisp(0:NPROC-1)) allocate(jlen(0:NPROC-1)) do irank=0,NPROC-1 CALL para_range(1,NENS,NPROC,irank,jsta,jend) jlen(irank)=jend-jsta+1 jdisp(irank)=jsta-1 end do CALL para_range(1,NENS,NPROC,MPIRANK,jsta,jend) #ifdef MPI_USE CALL MPI_BARRIER(MPI_COMM_WORLD,IERR) #endif N_LOC=jlen(MPIRANK)

Download ppt "DoD Center for Geosciences/Atmospheric Research at Colorado State University WSMR November 19-20, 2003 ARMY Research Lab and CIRA/CSU Collaboration on."

Similar presentations

Pattern Recognition and Machine Learning

Pattern Recognition and Machine Learning
Introduction to Data Assimilation NCEO Data-assimilation training days 5-7 July 2010 Peter Jan van Leeuwen Data Assimilation Research Center (DARC) University.

Introduction to Data Assimilation NCEO Data-assimilation training days 5-7 July 2010 Peter Jan van Leeuwen Data Assimilation Research Center (DARC) University.
Ensemble data assimilation (EnsDA) activities of the GOES-R project Progress report Dusanka Zupanski CIRA/CSU GOES-R meeting 8 September 2004 Dusanka Zupanski,

Ensemble data assimilation (EnsDA) activities of the GOES-R project Progress report Dusanka Zupanski CIRA/CSU GOES-R meeting 8 September 2004 Dusanka Zupanski,
P2.1 ENSEMBLE DATA ASSIMILATION: EXPERIMENTS USING NASA’S GEOS COLUMN PRECIPITATION MODEL D. Zupanski 1, A. Y. Hou 2, S. Zhang 2, M. Zupanski 1, C. D.

P2.1 ENSEMBLE DATA ASSIMILATION: EXPERIMENTS USING NASA’S GEOS COLUMN PRECIPITATION MODEL D. Zupanski 1, A. Y. Hou 2, S. Zhang 2, M. Zupanski 1, C. D.
Dusanka Zupanski CIRA/Colorado State University Fort Collins, Colorado Model Error and Parameter Estimation Joint NCAR/MMM CSU/CIRA Data Assimilation Workshop.

Dusanka Zupanski CIRA/Colorado State University Fort Collins, Colorado Model Error and Parameter Estimation Joint NCAR/MMM CSU/CIRA Data Assimilation Workshop.
Critical issues of ensemble data assimilation in application to GOES-R risk reduction program D. Zupanski 1, M. Zupanski 1, M. DeMaria 2, and L. Grasso.

Critical issues of ensemble data assimilation in application to GOES-R risk reduction program D. Zupanski 1, M. Zupanski 1, M. DeMaria 2, and L. Grasso.
1B.17 ASSESSING THE IMPACT OF OBSERVATIONS AND MODEL ERRORS IN THE ENSEMBLE DATA ASSIMILATION FRAMEWORK D. Zupanski 1, A. Y. Hou 2, S. Zhang 2, M. Zupanski.

1B.17 ASSESSING THE IMPACT OF OBSERVATIONS AND MODEL ERRORS IN THE ENSEMBLE DATA ASSIMILATION FRAMEWORK D. Zupanski 1, A. Y. Hou 2, S. Zhang 2, M. Zupanski.
P1.8 QUANTIFYING AND REDUCING UNCERTAINTY BY EMPLOYING MODEL ERROR ESTIMATION METHODS Dusanka Zupanski Cooperative Institute for Research in the Atmosphere.

P1.8 QUANTIFYING AND REDUCING UNCERTAINTY BY EMPLOYING MODEL ERROR ESTIMATION METHODS Dusanka Zupanski Cooperative Institute for Research in the Atmosphere.
Model error estimation employing ensemble data assimilation Dusanka Zupanski and Milija Zupanski CIRA/Colorado State University, Fort Collins, CO, U.S.A.

Model error estimation employing ensemble data assimilation Dusanka Zupanski and Milija Zupanski CIRA/Colorado State University, Fort Collins, CO, U.S.A.
MODEL ERROR ESTIMATION Cooperative Institute for Research in the Atmosphere Research Benefits to NOAA:This is a novel research approach, providing an optimal.

MODEL ERROR ESTIMATION Cooperative Institute for Research in the Atmosphere Research Benefits to NOAA:This is a novel research approach, providing an optimal.
Chapter 9 Approximating Eigenvalues

Chapter 9 Approximating Eigenvalues
Report of the Q2 Short Range QPF Discussion Group Jon Ahlquist Curtis Marshall John McGinley - lead Dan Petersen D. J. Seo Jean Vieux.

Report of the Q2 Short Range QPF Discussion Group Jon Ahlquist Curtis Marshall John McGinley - lead Dan Petersen D. J. Seo Jean Vieux.
Assimilation Algorithms: Tangent Linear and Adjoint models Yannick Trémolet ECMWF Data Assimilation Training Course March 2006.

Assimilation Algorithms: Tangent Linear and Adjoint models Yannick Trémolet ECMWF Data Assimilation Training Course March 2006.
The Inverse Regional Ocean Modeling System:

The Inverse Regional Ocean Modeling System:
Initialization Issues of Coupled Ocean-atmosphere Prediction System Climate and Environment System Research Center Seoul National University, Korea In-Sik.

Initialization Issues of Coupled Ocean-atmosphere Prediction System Climate and Environment System Research Center Seoul National University, Korea In-Sik.
Ibrahim Hoteit KAUST, CSIM, May 2010 Should we be using Data Assimilation to Combine Seismic Imaging and Reservoir Modeling? Earth Sciences and Engineering.

Ibrahim Hoteit KAUST, CSIM, May 2010 Should we be using Data Assimilation to Combine Seismic Imaging and Reservoir Modeling? Earth Sciences and Engineering.
Performance Optimization

Performance Optimization
280 SYSTEM IDENTIFICATION The System Identification Problem is to estimate a model of a system based on input-output data. Basic Configuration continuous.

280 SYSTEM IDENTIFICATION The System Identification Problem is to estimate a model of a system based on input-output data. Basic Configuration continuous.
1/20 Accelerating minimizations in ensemble variational assimilation G. Desroziers, L. Berre Météo-France/CNRS (CNRM/GAME)

1/20 Accelerating minimizations in ensemble variational assimilation G. Desroziers, L. Berre Météo-France/CNRS (CNRM/GAME)
A Concept of Environmental Forecasting and Variational Organization of Modeling Technology Vladimir Penenko Institute of Computational Mathematics and.

A Concept of Environmental Forecasting and Variational Organization of Modeling Technology Vladimir Penenko Institute of Computational Mathematics and.

Similar presentations

Ads by Google