Ting-Chi Wu CIRA Data Assimilation Group, CSU 5/9/2016 Adding New Observations in GSI: type, operator, TLM/ADJ code, diagnosis and statistics Ting-Chi Wu CIRA Data Assimilation Group, CSU 5/9/2016 For questions and comments. Please send to: Ting-Chi.Wu@colostate.edu

List of codes that require changes based on GSI v3.4 read_obs.F90 read_NEWbufr.f90 m_obdiag.F90 obsmod.F90 read_obsdiag.F90 write_obsdiag.F90 setuprhsall.f90 setupNEW.f90 statsNEW.f90☆ (or use statsconv.f90) intjo.f90 intNEW.f90 stpjo.f90 stpNEW.f90 obs_sensitivity.f90 setupyobs.f90 adjtest_obs.f90 makefile_DTC Makefile.dependency

List of codes that require changes based on GSI v3.4 read_obs.F90 read_NEWbufr.f90☆ m_obdiag.F90 obsmod.F90 read_obsdiag.F90 write_obsdiag.F90 setuprhsall.f90 setupNEW.f90☆ statsNEW.f90☆ (or use statsconv.f90) intjo.f90 intNEW.f90☆ stpjo.f90 stpNEW.f90☆ obs_sensitivity.f90 setupyobs.f90 adjtest_obs.f90 makefile_DTC Makefile.dependency ☆ NEW

Observations Assimilated in GSI read_obs. F90 (p Observations Assimilated in GSI read_obs.F90 (p.23 of Advanced GSI User’s Guide) Satellite radiance (*bufr): HIRS, MHS, AMSU-A, AIRS, IASI, GOES, ATMS, etc read_bufrtovs.f90, read_airs.f90, read_iasi.f90, read_goesndr.f90, read_atms.f90, etc Conventional observations (prepbufr, radarbufr, tcvitl, etc): radiosonde, dropwindsonde, aircraft reports, surface ship and buoy obs, surface obs overland, pibal winds, wind profiler, satellite derived winds, TPW, radar winds, TCVitals, etc read_prepbufr.f90, read_rw.f90, read_tcps.f90, etc

Observation Files in GSI *bufr and gsiparm.nml The format is BUFR (Binary Universal Form for the Representation of Meteorological data) Satellite radiances: atmsbufr, airsbufr, etc Conventional observations: prepbufr (stands for NCEP “prepared” or “QCed” BUFR), radarbufr, tcvitl, satwndbufr, etc. Check GSI/util/bufr_tools/ Sample codes to decode/encode/append observation info from/to a prepbufr file.

GSI Code Structure xa = Δxouter_iteration + Δxinner_inneration + xb In outer iteration: glbsoi.f90 (jiter=1,miter) Full (nonlinear) operator h (setuprhsall calls setupXXX) Quality control (included in setupXXX) In inner iteration: pcgsoi.f90 (iter=1,niter(jiter)) Preconditioned cost function minimization - algorithm 1: preconditioned conjugate gradient (pcg in short) Estimate search direction Estimate step size in search direction (stpcalc calls stpXXX) Simpler (linear) operator H and HT (intjo calls intXXX) Variational quality control (included in intXXX) Solution used to start the next outer iteration

Observations Operators in GSI - I setuprhsall.f90 setuprhsall calls a set of subroutines (setupXXX) that calculate y-h(x) for for satellite radiances, conventional observations, ozone, chemistry, etc using their corresponding operators: Satellite radiances (Tb): setuprad.f90 (call CRTM through crtm_interface.f90) Conventional observations (t, ps, q, u, v, etc): setupt.f90, setupps.f90, setupq.f90, setupw.f90, etc. setuprhsall is called by glbsoi.f90 in each outer iteration.

Observations Operators in GSI - II intjo.f90 intjo also calls a set of subroutines (intXXX) that compute tangent linear and adjoint operators for each type of observations : Satellite radiance: intrad.f90 Conventional observations: intt.f90, intps.f90, intq.f90, intw.f90, etc. intjo is called by intall.f90, which is called in each inner iteration by pcgsoi.f90.

Stepsize Computation in GSI stpjo.f90 Similarly, stpjo calls a collection of subroutines (stpXXX) that compute step size for each type of observations: Satellite radiance: stprad.f90 Conventional observations: stpt.f90, stpps.f90, stpq.f90, stpw.f90, etc. stpjo is called by stpcalc.f90, which is called in each inner iteration by pcgsoi.f90 to conduct cost function minimization.

Step-by-step coding instruction Will use NEW as the new observation name For example, NEW can be lwp (i.e. liquid water path). My personal suggestion: find a similar observation that is already used by GSI and follow it. For example, lwp is similar to pw (i.e. total precip water) Notations for the following slides: bold and italic font: Fortran programs that require changes; ☆: new Fortran programs to be added blue and italic: the actual coding that one has to add into the corresponding Fortran programs. Underline: indicates a file.

List of codes that require changes based on GSI v3.4 read_obs.F90 read_NEWbufr.f90☆ m_obdiag.F90 obsmod.F90 read_obsdiag.F90 write_obsdiag.F90 setuprhsall.f90 setupNEW.f90☆ statsNEW.f90☆ (or use statsconv.f90) intjo.f90 intNEW.f90☆ stpjo.f90 stpNEW.f90☆ obs_sensitivity.f90 setupyobs.f90 adjtest_obs.f90 makefile_DTC Makefile.dependency

read_obs.F90: read_NEWbufr.f90☆: Add check if NEWbufr file exists. If exists, check if NEW obstype is used according to gsiparm.anl and NEWinfo or convinfo table Add call read_NEWbufr read_NEWbufr.f90☆: Read observation values, locations (lat, lon, and/or height/pressure), times, station ID, errors, quality flags, types, usages, etc Save the above info in a file (obs_input.00xx) to be used later

List of codes that require changes based on GSI v3.4 read_obs.F90 read_NEWbufr.f90☆ m_obdiag.F90 obsmod.F90 read_obsdiag.F90 write_obsdiag.F90 setuprhsall.f90 setupNEW.f90☆ statsNEW.f90☆ (or use statsconv.f90) intjo.f90 intNEW.f90☆ stpjo.f90 stpNEW.f90☆ obs_sensitivity.f90 setupyobs.f90 adjtest_obs.f90 makefile_DTC Makefile.dependency

m_obdiag.F90: obsmod.F90: Add NEW_verify_ function Add i_NEW_ob_type Add and define NEW_ob_type and NEW_ob_head types and associated pointers (NEW, NEWhead and NEWtail) Add iout_NEW (fort.2**), mype_NEW, and cobtype(i_NEW_ob_type)=“NEW” Add corresponding pointer association in subroutine destroyobs_

read_obsdiag.F90: write_obsdiag.F90: Add use obsmod, only: i_NEW_ob_type Add subroutine read_NEWhead_ write_obsdiag.F90: Add subroutine write_NEWhead_ Both subroutines read_NEWhead_ and write_NEWhead_ need to be coded following the structure and types of NEW_ob_type and NEW_ob_head that are defined in obsmod.F90

List of codes that require changes based on GSI v3.4 read_obs.F90 read_NEWbufr.f90☆ m_obdiag.F90 obsmod.F90 read_obsdiag.F90 write_obsdiag.F90 setuprhsall.f90 setupNEW.f90☆ statsNEW.f90☆ (or use statsconv.f90) intjo.f90 intNEW.f90☆ stpjo.f90 stpNEW.f90☆ obs_sensitivity.f90 setupyobs.f90 adjtest_obs.f90 makefile_DTC Makefile.dependency

create statsNEW.f90☆ or use statsconv.f90 : setuprhsall.f90: Add i_NEW Add Check if NEW obstype is used Add call setupNEW Add call statsNEW (or statsconv) create statsNEW.f90☆ or use statsconv.f90 : Add use obsmod, only: i_out_NEW, mype_NEW Write statistics (bias, rmse, penality, etc) of NEW observations into the corresponding fort.2XX file (where XX is a number indicated by i_out_NEW) after each outer loop.

setupNEW.f90☆: Calculate guess values using the operator (nonlinear) Read obs_input.00xx file Interpolate guess values to observation space and compute innovation (fit to observation; i.e. y-h(x)) Compute Jacobians and interpolation coefficients, and associate them with the corresponding pointer (NEWtail). The pointer and the information carried by the pointer will be used in intNEW.f90 and stpNEW.f90 Save fit to observation information (lat, lon, time, height, innovation, obs value, etc) to diag file (e.g. diag_NEW_ges/diag_NEW_anl), if specified in gsiparm.anl.

List of codes that require changes based on GSI v3.4 read_obs.F90 read_NEWbufr.f90☆ m_obdiag.F90 obsmod.F90 read_obsdiag.F90 write_obsdiag.F90 setuprhsall.f90 setupNEW.f90☆ statsNEW.f90☆ (or use statsconv.f90) intjo.f90 intNEW.f90☆ stpjo.f90 stpNEW.f90☆ obs_sensitivity.f90 setupyobs.f90 adjtest_obs.f90 makefile_DTC Makefile.dependency

intjo.f90: intNEW.f90☆: Add call intNEW Associate NEWptr with NEWtail Read in sval pointers of corresponding state variables (e.g., tsen, pres, u, v, ect) from gsi_bundlegetpointer Run tangent linear operator of NEW using information carried by NEWptr pointer Apply nonlinear quality control Run adjoint operator of NEW and the output is: rval pointers of corresponding state variables

stpjo.f90: stpNEW.f90☆: Add call stpNEW Similar to intNEW.f90, except that the adjoint operator is not required here.

List of codes that require changes based on GSI v3.4 read_obs.F90 read_NEWbufr.f90☆ m_obdiag.F90 obsmod.F90 read_obsdiag.F90 write_obsdiag.F90 setuprhsall.f90 setupNEW.f90☆ statsNEW.f90☆ (or use statsconv.f90) intjo.f90 intNEW.f90☆ stpjo.f90 stpNEW.f90☆ obs_sensitivity.f90 setupyobs.f90 adjtest_obs.f90 makefile_DTC Makefile.dependency

obs_sensitivity.f90: setupyobs.f90: adjtest_obs.f90: Add use obsmod, only: i_NEW_ob_type Add cobtype(i_NEW_ob_type)=“NEW” setupyobs.f90: Add yobs(ii)%NEWptr association with NEWhead(ii)%head adjtest_obs.f90: Add NEWptr under NEW_ob_type Add NEWptr association with yobs%NEWptr Add lhs and nob summation for NEWptr Add NEWptr association with NEWptr%llpoint Add call mpi_allreduce

List of codes that require changes based on GSI v3.4 read_obs.F90 read_NEWbufr.f90☆ m_obdiag.F90 obsmod.F90 read_obsdiag.F90 write_obsdiag.F90 setuprhsall.f90 setupNEW.f90☆ statsNEW.f90☆ (or use statsconv.f90) intjo.f90 intNEW.f90☆ stpjo.f90 stpNEW.f90☆ obs_sensitivity.f90 setupyobs.f90 adjtest_obs.f90 makefile_DTC Makefile.dependency

makefile_DTC: Makefile.dependency: Add .o of the newly added codes (☆): For example, read_NEWbufr.o, setupNEW.o, statsNEW.o, intNEW.o, and stpNEW.o Makefile.dependency: Add .o of the newly added codes (☆) and their dependency; any modules that were utilized in the new codes through the use statement For example, setupNEW.o : setupNEW.f90 gridmod.o obsmod.o kinds.o guess_grid.o … etc

Finally… compile and run Once you have made the above-mentioned changes, you should be ready to re-compile GSI. ./compile |& tee compile.log and correct errors (Err) Single observation experiments are recommended: One can create a BUFR file that contains a single NEW observation using the GSI/util/bufr_tools/ sample codes. And then, examine the response of assimilating a single NEW observation on state/control variables and nearby grid points. Don’t forget to add NEW in the &OBS_INPUT section in your gsiparm.anl namelist. Also, add changes to convinfo, satinfo, or create newinfo to assign the usage, observation errors, thinning option, etc of the NEW observation you are going to assimilate. Depending on the design of the NEW operator, changes to anavinfo may be required as well.

Useful Guides GSI User’s Guide and Advanced User’s Guide: http://www.dtcenter.org/com-GSI/users/docs/index.php BUFR/PrepBUFR User’s Guide: ftp://ftp.cpc.ncep.noaa.gov/hwang/HW/BUFR/BUFR_PrepBUFR_User_Guide_v1.pdf Ricardo Todling’s Talk on GSI Infrastructure: http://www.dtcenter.org/com-GSI/users/docs/presentations/2015_tutorial/20150813_L11_Todling_GSI_Infrastructures.pdf