Potpourri ATM 419/563 Spring 2017 Fovell.

Potpourri ATM 419/563 Spring 2017 Fovell

Outline Making animations with GrADS
Downloading and processing MADIS data Verifying against radiosonde observations Directly modifying WRF/WPS output files with read_wrf_nc.f Some sources of initialization data Adaptive time stepping Writing out extra fields to wrfout or other output files Restarting WRF simulations Using nestdown for 1-way nesting Reference is made to the “SCRIPTS” directory: /network/rit/lab/atm419lab/SCRIPTS

Making animations with GrADS

say 'Create PNG images as well (1=yes ; 0=no)' pull ans frame = 1
'q file' rec=sublin(result,5) _endtime=subwrd(rec,12) runscript = 1 * Set to first time desired dis_t = 1 while(runscript) 'set t ' dis_t 'c' 'set grads off' * * GRADS PLOTTING COMMANDS HERE dis_t set to initial time desired

* --------------------------------------------------------
* GRADS PLOTTING COMMANDS HERE if(ans) if(frame < 10) 'printim movie_00'frame'.png png ' else if(frame < 100) 'printim movie_0'frame'.png png ' 'printim movie_'frame'.png png ' endif frame=frame+1 *pull dummy if ( dis_t=_endtime ) runscript=0 dis_t = dis_t + 1 endwhile Individual frames called movie_XXX.png Remove “*” to require hitting return key between frames dis_t updated by one to plot next time

Merging PNG files into animated GIF
The script will result in a sequence of files called movie_XXX.png Combine these into a single, animated GIF file using convert Change delay to alter frame rate Loop “0” is endless loop; “1” plays only once convert -delay 30 -loop 0 movie*png OUT.GIF

2D squall line experiment movie
reflec_movie.gs in SCRIPTS/

Why animated GIFs? Well-supported format, works in all browers
Open and edit in applications like Preview and GraphicConverter (Mac) Remove or rearrange frames, adjust frame rate, crop the entire movie… They work well with PowerPoint Linking external animations to PowerPoint presentations are prone to glitches, including broken links Trick: Insert an animated GIF as a “photo”, not as a “movie”. The animation then becomes incorporated into the presentation.

Downloading and processing MADIS surface data

Scripts MADIS_download.sh MADIS2ASCII.sh (and MADIS2ASCII.pl)
ASCII2NC.sh [There now is a MADIS2NC program that can skip the MADIS2ASCII/ASCII2NC step, but there are useful data filtering options in the MADIS2ASCII step.]

MADIS_download.sh By default, MADIS_download.sh will download ALL surface data for the entire world (see next slide), which takes time. DIR=${PWD} Date_base= Date_base_hour=00 run_hours=54 […] echo "will get data from "$date"_"$date_hr" to "$date2"_"$date2_hr echo "Is this correct (y/n)?" read input Downloads data to current directory Starting 9/1/2016 at 00Z …and for 54 hours additional hours

All available MADIS surface stations for a selected hour
N = 91,501 entries Z

MADIS_download.sh This is the key line in the script:
As configured, this will download all fields from available stations, from all data providers, at the specified time AND over the previous 59 min, independent of quality control flag setting The script specifically asks for the “cprk” server which has a more extensive archive Downloading can take a long time! site=" madisXmlPublicDir?rdr=&time="$date"_"$date_hr"00&minbck=-59&minfwd=0& recwin=4&dfltrsel=2&latll=0.0&lonll=0.0&latur=90.0&lonur=0.0& stanam=&stasel=0&pvdrsel=0&varsel=2&qcsel=0&xml=0&csvmiss=0”

MADIS_download.sh site=" madisXmlPublicDir?rdr=&time="$date"_"$date_hr"00&minbck=-59&minfwd=0 &recwin=4&dfltrsel=2&latll=0.0&lonll=0.0&latur=90.0&lonur=0.0 &stanam=&stasel=0&pvdrsel=0&varsel=2&qcsel=0&xml=0&csvmiss=0” • Time window extends ahead 0 minutes and behind 50 minutes (&minbck=-50 and &minfwd=0) Alternative: &minbck=-30 and &minfwd=30 (requested time ± 30 min) • All observations, irrespective of quality control flag (&qcsel=0) Alternative: &qcsel=1 • Returns all records found in time interval requested (&recwin=4) Alternatives: &recwin=1 - return one record per station, closest to analysis time &recwin=2 - return one record per station, closest to start of time window &recwin=3 - return one record per station, closest to end of time window

MADIS_download.sh • More options
site=" madisXmlPublicDir?rdr=&time="$date"_"$date_hr"00&minbck=-59&minfwd=0& recwin=4&dfltrsel=2&latll=0.0&lonll=0.0&latur=90.0&lonur=0.0& stanam=&stasel=0&pvdrsel=0&varsel=2&qcsel=0&xml=0&csvmiss=0” • More options - Set latitude and longitude limits (&latll, &latur, &lonll, &lonur for values at lower left (ll) and upper right (ur) of specific area - &state=CA (would limit request to stations in California) - &pvdrsel=1 and &pvd=ASOS (would activate provider filter and limit data obtained to ASOS stations) - &varsel=1 and &nvars=T&nvars=TD&nvars=RH&nvars=DD&nvars=FF&nvars=FFGUST (limit data obtained to temperature, dewpoint, RH, wind direction, wind speed and gust)

MADIS2ASCII.sh, MADIS2ASCII.pl
These scripts do additional processing on the raw MADIS data In particular, MADIS2ASCII.pl is currently configured to retain only ASOS station data, for a selection of fields (but can be edited) Also contains optional code to filter by elevation, lat/lon There is a MADIS2NC tool now, untried – as my script does some potentially useful filtering #station type filter $typefilter=1; if ( $station_type eq 'ASOS' ){

Sequence Edit MADIS_download.sh
sh MADIS_download.sh Downloads files like _00.mtxt Make sure MADIS2ASCII.pl is configured as you need sh MADIS2ASCII.sh Creates files like _00.out sh ASCII2NC.sh Creates files like _00.nc Edit MET_run_ASCII2.sh so OBS_base points to your MADIS NetCDF files OBS_base=$MYLAB/MADIS/MADIS_ _ASOS

Verifying against radiosonde data

Upper air data: PREPBUFR archive
Need a (free) account on NCAR’s Research Data Archive ( Direct link to PREPBUFR data: You will obtain the data as tar files with names like prepbufr nr.tar.gz. After unpacking (with tar –zxvf), they’ll have file extension “.nr” $ tar –zxvf prepbufr nr.tar.gz sh PREP2NC.sh will operate on .nr files, creating NetCDF files. This can take a long time.

Example: KANSAS01 case Move or remove KANSAS01/postprd/point_stat* files (created from analysis of ASOS surface stations) Copy MET_run_prepbufr.sh from SCRIPTS directory into KANSAS01 Edit and execute MET_run_prepbufr.sh [script is preconfigured for KANSAS01 case] Compares T and wind at 925, 850, 700, and 500 mb 1000 mb level skipped since it is not always represented in dataset plot_met.sh NN NN={2,..5} for T at p={925,…,500 mb} NN={6,..,9} for wind speed at p={925,…,500 mb}

MET_run_prepbufr.sh Date_base=20160313 Date_hour=00 domain=1
MYHOME=/network/rit/home/rf518462 MYLAB=/network/rit/lab/fovelllab_rit MET_base=$MYHOME/METV5 OBS_base=/network/rit/lab/atm419lab/PREPBUFR/PREPBUFR_ RUN_base=$PWD/postprd

[remainder of the output lines shown above..]
Forecast Observ N $ sh plot_met.sh 2 _ TMPf TMPo N 14 MAE BCRMSE _ TMPf TMPo N 14 MAE BCRMSE _ TMPf TMPo N 13 MAE BCRMSE _ TMPf TMPo N 14 MAE BCRMSE _ TMPf TMPo N 14 MAE BCRMSE [remainder of the output lines shown above..] MSE BIAS STDf STDo LVL P925f P925o MSE BIAS STDf STDo LVL P925f P925o MSE BIAS STDf STDo LVL P925f P925o MSE BIAS STDf STDo LVL P925f P925o MSE BIAS STDf STDo LVL P925f P925o Make sure forecast level = observation level (the script can get fooled by problems in the output) Check N and the fields being compared at each time. Missing data can make for odd statistics.

925 mb temperature (average of 13-14 radiosondes)

MAE by forecast hour and pressure level
MAE quickly jumps at 925 mb between 0-12 h and remains high. A LSM issue?

Modifying WRF/WPS files: read_wrf_nc.f
[Alternatives using NCL or python can be persued]

read_wrf_nc.f This program reads in WRF/WPS-generated files, and modifies them as you specify. Examples: you can modify real.exe or ideal.exe output (e.g., wrfinput_d01), geogrid.exe output (e.g., geo_em.d01.nc) Specific example: add passive tracers in real or idealized experiments You can easily add up to 8 passive tracer fields Get from SCRIPTS directory: compile_real.csh, read_wrf_nc.f

Example: 2D idealized squall line
The read_wrf_nc.f program is pre-configured to add a new passive tracer field called “tr17_1”, concentrated near the surface in our 2D squall line experiment Create a directory called SQUALL2, and copy SETUP.tar from /network/rit/lab/atm419lab/SQUALL/ directory Unpack and execute make_all_links.csh Modify control_file.all to insert this new line in the 3D fields section (see next slide) tr17_ ! Tracer 1 Edit namelist.input to insert this new line in the &dynamics section (see next slide): tracer_opt = 2 Execute ideal.exe to create wrfinput_d01

Modified files • modified namelist.input • modified control_file.all
&dynamics tracer_opt = 2, rk_ord = 3, diff_opt = 2, , , km_opt = 2, , , damp_opt = 2, • modified control_file.all TD ! Dewpoint Temperature (diagnostic) RH ! Relative Humidity (diagnostic) tr17_ ! tracer 1 H_DIABATIC ! h_diabatic Specific locations not important, as long as they’re in the correct sections

Modifying wrfinput_d01 Edit read_wrf_nc.f as needed Compile it
(for this example, you can skip this step) Compile it csh compile_read.csh Execute it ./read_wrf_nc.exe wrfinput_d01 -EditData tr17_1 Run wrf.exe, use wrf_to_grads with control_file.all Tracer field is called “tr17_1”

Example tracer code in read_wrf_nc.f
if (var == 'tr17_1’ .or. var == 'tr17_2’)then do xx=1,dim1 do yy=1,dim2 do zz=1,dim3 if(var == 'tr17_1') then if(zz.gt.5)then data_real(xx,yy,zz) = 0.0 else data_real(xx,yy,zz) = 1.0 endif enddo Tracer = 1.0 for levels 1-5, inclusive, for all x

Try step.gs tr17_1-tr17_1(t=1) 2 7 0.2
Watch the tracer lofted into the storm, and depleted from the cold pool set t 1 set x 1 d tr17_1

U and tracer at final time

Example: modify SOC topography
if ( var == 'TSK') then ! Force TSK values to data_real = elseif ( var == 'SOILHGT') then ! raise soil height by 30% where (data_real .gt. 0.0) data_real = data_real + .3*data_real endwhere elseif ( var == 'ISLTYP') then ! change all 1's to 2's where (data_int == 1 ) data_int = 2 elseif ( var == 'LU_INDEX') then data_real=7 elseif ( var == 'HGT_M') then ! cut topo height by 50 pct. do xx=1,dim1 do yy=1,dim2 data_real(xx,yy,1) = data_real(xx,yy,1)*0.5 enddo endif HGT_M is a real (non-integer field) read it in, then cut by 50% Part of read_wrf_nc.f

Example: modify SOC topography
Modify read_wrf_nc.f, compile, execute on geogrid output file ./read_wrf_nc.exe geo_em.d01.nc -EditData HGT_M Since geogrid output has been altered, run metgrid.exe and real.exe AGAIN. No need to rerun ungrib.exe.

Input data sources

Some sources for initialization data
NCAR RDA (free account required) NCEP NOMADS server (for recent data) The get_gfs.csh script in /network/rit/lab/atm419lab/SCRIPTS is configured to fetch the most recent 00Z GFS 0.25˚ grids via FTP. It can be modified to fetch different datasets. NCDC NOMADS server (for older data)

requires approval requires approval (analysis) (analysis) see slide 41 From WRF tutorial presentation

From WRF tutorial presentation

Unidata Internet Data Distribution (IDD) archive at rda.ucar.edu
ds335.0 • GFS 0.5˚ data are usable as-is in WPS (tho they are 6-h analyses, not forecasts) [see ds084.1 for 0.25˚ files for 2015-present] • NAM 12km files need a lot of work - analyses + forecasts combined - soil information is missing - ds609.0 is alternative, but analyses only • RAP 20km are also problematic

Adaptive time stepping
[can save a lot of time, but also can cause a lot of problems]

Modification to namelist.input (&domains section)
use_adaptive_time_step = .true., step_to_output_time = .true., target_cfl = 1.2, 1.2, target_hcfl = .84, .84, max_step_increase_pct = 5, 51, starting_time_step = -1, -1, max_time_step = 720, 720, min_time_step = 240, 240, adaptation_domain = 1, Preserves requested output times (for outermost domain anyway) Starts with ∆t = 4∆x (∆x in km) Often based on 8∆x and 3-4∆x Domain to use to adjust ∆t • CFL = (fastest wave speed)*∆t/(grid length) • increase time step if vertical CFL < target_cfl and horizontal CFL < target_hcfl • when use_adaptive_time_step = .true., time_step is ignored • Typically use a larger max_step_increase in nests than in outer domain (as shown here)

Adaptive time stepping for SOC case
Created from information written to rsl.out.XXXX files

NNRP-based SOC runs sensitivity to time stepping
RMS SLP error (NNRP) NESIS adaptive 2.01 mb/gridpt 10.93 ∆t = 720 sec 2.04 11.09 ∆t = 540 sec 2.02 11.37 ∆t = 240 sec 2.00 11.35 ∆t = 120 sec 11.11 • Microphysics is more sensitive to time step than many other model physics and numerics (such as advection)

Outputting extra fields

Outputting extra fields
By default, a lot of fields computed or used by WRF are NOT written to the wrfout files These can be added to the wrfout files by Editing the appropriate WRF Registry (e.g., Registry.EM_COMMON) file and recompiling from scratch (after clean –a) Using the “iofields” option in namelist.input Example: Write out boundary layer tendencies RUBLTEN, RVBLTEN, RTHBLTEN, and RQVBLTEN

Modify namelist.input (&time_control section)
Add these lines: Contents of “my_file_d01.txt”: These fields have units of Pa (m/s2), Pa (K/s), and Pa (kg/kg/s), so have to be divided by (MU+MUB) [total dry air mass in column] iofields_filename = "my_file_d01.txt","my_file_d02.txt","my_file_d03.txt", ignore_iofields_warning = .true., +:h:0:RUBLTEN,RVBLTEN,RTHBLTEN,RQVBLTEN

Extra fields with more frequent outputs (&time_control section)
Add two more lines: Alter “my_file_d01.txt”: Now these extra fields are written to a file called auxhist24_d01… instead, and at (possibly) its own history interval (here, 180 min) iofields_filename = "my_file_d01.txt","my_file_d02.txt","my_file_d03.txt", ignore_iofields_warning = .true., io_form_auxhist24 = 2, auxhist24_interval = 180, +:h:24:RUBLTEN,RVBLTEN,RTHBLTEN,RQVBLTEN

Restarting WRF

In namelist.input Our SOC case started on 1993-3-12-12
Suppose you had restart_interval = This would make restart files (called wrfrst_d…) every 720 min or 12 h. To restart the model at : Revise start_day, start_hour (to restart time) Revise run_days, run_hours (to be shorter) Set restart = .true. Model will NOT write a wrfout file at restart time unless you also add this line: write_hist_at_0h_rst = .true.

Nestdown

Nestdown We have made simulations with nested domains in which the nests are integrated at the same time as the outermost domain. The initial and boundary conditions came from a different parent model (GFS, NARR, etc.). This can be expensive, particularly if the sole purpose of the outer domain is to keep the lateral boundaries away from the nested region of interest The nestdown procedure uses a coarse resolution WRF simulation to provide initial and boundary conditions for a higher resolution WRF run. This is effectively 1-way nesting

2-domain configuration
for INDIANA case

Two-way vs. one-way nesting
When we have employed nests, we have been using two-way nesting Two-way nesting (for a 2-domain setup): Execute domain 1 (on time step ∆t) Execute domain 2 (probably on time step ∆t/3) At the end of time period ∆t, use domain 2 information to overwrite domain 1 at the shared grid points Two ways of doing one-way nesting Execute both domains simultaneously but prevent D2 from overwriting D1 information feedback option in &domains (0 = 1-way, 1 = 2-way) This is just as expensive as any other two-domain simulation Use nestdown Uses D1 information to initialize and drive D2 Not running D1 and D2 simulatenously

Nestdown procedure: 2-domain setup #1
Configure parent domain and nest using geogrid.exe (max_dom = 2) Run ungrib.exe as usual Execute metgrid.exe on the two domains Then, execute real.exe on ONE domain, and make your coarse resolution WRF run Consider history_interval of 60 min or shorter Retain wrfout_d01* file(s) in current directory

Execute real.exe again, but now with max_dom = 2 This produces wrfinput_d01 and wrfinput_d02 Rename wrfinput_d02 as wrfndi_d02 Edit namelist.input, to add this line to &time_control section: io_form_auxinput2 = 2, Also, change interval_seconds to 3600 (matching 60-min wrfout_d01* writes) These files are now going to be used as input: The wrfout_d01* file(s) from single domain run The wrfndi_d02 file

Link to ndown.exe program ln -sf /network/rit/home/atm419/WRFV371_ATM419/main/ndown.exe ndown.exe Submit submit_ndown to Snow Check rsl.out.0000 file for “ndown_em: SUCCESS COMPLETE NDOWN_EM INIT” This step creates new files wrfinput_d02 and wrfbdy_d02. Rename these mv wrfinput_d02 wrfinput_d01 mv wrfbdy_d02 wrfbdy_d01 This overwrites/replaces your original files. If you wish, please save copies of the originals somewhere

Also, remove or move the original wrfout_d01* files. Otherwise, they will be overwritten when you run wrf.exe again In addition, save a copy of your original namelist.input file cp namelist.input namelist.input.original With nestdown, your previous domain #2 is becoming your new domain #1

Again, your previous domain #2 is now your domain #1 Edit the namelist.input file. You now need max_dom = 1 For SOME entries, you need to COPY the contents of column 2 to column 1!!! This is because your Domain 2 is now your Domain 1 See practical example ALSO, change your time_step! Previously, ∆t for D2 was time_step/parent_time_step_ratio (likely time_step/3) Thus, your new time_step should likely be divided by 3 Your new run will write to wrfout_d01* BUT will represent output for your original D2

Optional example: INDIANA case (1/N)
Revert to GFS initialization I am presuming you still have ungribbed ‘GFS’ data, and 2 domains created using geogrid.exe in your INDIANA directory If not, start from scratch [see ATM419_WRF_real_ADVANCED.pptx] Two new files to copy from /network/rit/lab/atm419lab/INDIANA: make_all_links.csh [adds ndown.exe link] submit_ndown Execute make_all_links.csh Edit namelist.wps for GFS data interval_seconds = [for GFS] &ungrib prefix=‘GFS’ and &metgrid fg_name=‘GFS’ Make sure max_dom = 2 Execute metgrid.exe

Make your single-domain GFS run again, if necessary. (If you saved your wrfout_d01* files from this simulation, with 60-min histories, you need not rerun it.) Edit namelist.input for GFS: interval_seconds num_metgrid_levels max_dom = 1 history_interval = 60 [hourly outputs, at a maximum] Execute real.exe Submit WRF job. Keep wrfout_d01* outputs in current directory Next step: set max_dom = 2 in namelist.input and rerun real.exe. This yields wrfinput_d01 and wrfinput_d02. Rename wrfinput_d02 as wrfndi_d02

Edit namelist.input yet again. Add this line to &time_control io_form_auxinput2 = 2, Also, set interval_seconds = 3600 [ = your 60-min history_interval] Submit submit_ndown to Snow. Upon successful completion, nestdown produces these files: wrfinput_d02 and wrfbdy_d02 Move or remove your wrfout_d01* file(s) Rename the newly created files mv wrfinput_d02 wrfinput_d01 mv wrfbdy_d02 wrfbdy_d01 Copy the namelist.input as namelist.input.original Edit namelist.input to shift D2 information to D1 See next slides

&time_control io_form_auxinput2 = 2, run_days = 0, run_hours = 12, run_minutes = 0, run_seconds = 0, start_year = 2016, 2016, start_month = 03, 03, start_day = 27, 27, start_hour = 12, 12, start_minute = 00, 00, start_second = 00, 00, end_year = 2016, 2016, end_month = 03, 03, end_day = 28, 28, end_hour = 00, 00, end_minute = 00, 00, end_second = 00, 00, interval_seconds = 10800, input_from_file = .true., .true., history_interval = 60, 20, Original namelist.input

&time_control io_form_auxinput2 = 2, run_days = 0, run_hours = 12, run_minutes = 0, run_seconds = 0, start_year = 2016, 2016, start_month = 03, 03, start_day = 27, 27, start_hour = 12, 12, start_minute = 00, 00, start_second = 00, 00, end_year = 2016, 2016, end_month = 03, 03, end_day = 28, 28, end_hour = 00, 00, end_minute = 00, 00, end_second = 00, 00, interval_seconds = 3600, input_from_file = .true., .true., history_interval = 20, 20, MODIFIED namelist.input • don’t need that io_form statement any more • interval_seconds to match coarse grid history interval (60 min = 3600s ) • may want to reconsider your output interval Should have been 3600 for ndown

&domains time_step = 90, time_step_fract_num = 0, time_step_fract_den = 1, max_dom = 2, e_we = 201, 214, e_sn = 201, 196, e_vert = 57, 57, p_top_requested = 5000, num_metgrid_levels = 27, num_metgrid_soil_levels = 4, dx = , , dy = , , grid_id = 1, 2, parent_id = 0, 1, i_parent_start = 1, , j_parent_start = 1, , parent_grid_ratio = 1, 3, parent_time_step_ratio = 1, 3, Original namelist.input

&domains time_step = 30, time_step_fract_num = 0, time_step_fract_den = 1, max_dom = 1, e_we = 214, 214, e_sn = 196, 196, e_vert = 57, 57, p_top_requested = 5000, num_metgrid_levels = 27, num_metgrid_soil_levels = 4, dx = 4000., , dy = 4000., , grid_id = 1, 2, parent_id = 0, 1, i_parent_start = 1, , j_parent_start = 1, , parent_grid_ratio = 1, 3, parent_time_step_ratio = 1, 3, MODIFIED namelist.input • decreased time_step • now only one domain! • shifted D2 e_we and e_sn to first column • sshifted dx, dy for D2 to first column

&physics mp_physics = 8, 8, ra_lw_physics = 4, 4, ra_sw_physics = 4, 4, radt = 20, 20, sf_sfclay_physics = 1, 1, sf_surface_physics = 2, 2, bl_pbl_physics = 1, 1, bldt = 0, 0, cu_physics = 1, 0, Original namelist.input

&physics mp_physics = 8, 8, ra_lw_physics = 4, 4, ra_sw_physics = 4, 4, radt = 20, 20, sf_sfclay_physics = 1, 1, sf_surface_physics = 2, 2, bl_pbl_physics = 1, 1, bldt = 0, 0, cu_physics = 0, 0, MODIFIED namelist.input • Turned off cumulus as not wanted at 4 km • Consider other physics changes Next slides: comparison of original 2-domain GFS with nestdown version. Ideally, the results should be the same, which would mean you don’t need to keep rerunning D1 all the time as you investigate changes for D2

nestdown interval_seconds = 3600

nestdown interval_seconds = 10800

Potpourri ATM 419/563 Spring 2017 Fovell.

Similar presentations

Presentation on theme: "Potpourri ATM 419/563 Spring 2017 Fovell."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Potpourri ATM 419/563 Spring 2017 Fovell.

Similar presentations

Presentation on theme: "Potpourri ATM 419/563 Spring 2017 Fovell."— Presentation transcript:

Similar presentations

About project

Feedback