WRFDA Regression Testing Suite Michael Kavulich August 21, 2014

WRFDA Regression Testing Suite The WRFDA Regression Testing Suite (RTS) is designed to detect errors in various parts of the WRFDA system, The testing system consists of the following: A main script (regtest.pl) A test data file (testdata.txt) A test database (WRFDA-data-EM) A baseline database (BASELINE.NEW)

WRFDA Regression Testing Suite You can get the scripts and text files from the following repository: https://svn-user-kavulich.cgd.ucar.edu/trunk/REGTEST/ Download it by using “svn checkout” svn checkout https://svn-user-kavulich.cgd.ucar.edu/trunk/REGTEST/ The test database and baseline database are very large, and must be downloaded from the website: http://www.mmm.ucar.edu/wrf/users/wrfda/regression/

What the test data file looks like [kavulich@yslogin4 REGTEST]$ vi testdata.txt ########################################################################################### #ARCH MACHINE NAME SOURCE COMPILER PROJECT QUEUE DATABASE BASELINE Linux x86_64 yellowstone WORKDIR/2014-08-13_changes/multi-level_and_gps_fixes/wrfda.tar ifort P64000400 caldera WRFDA-data-EM BASELINE.NEW #INDEX EXPERIMENT TYPE CPU OPENMP PAROPT 1 afwa_t7_ssmi 3DVAR 4 16 serial|dmpar 2 afwa_t7_ssmi_32 3DVAR 4 16 serial|dmpar 3 ASR_prepbufr 3DVAR 16 16 serial|dmpar 4 ASR_airs 3DVAR 16 16 dmpar 5 cv3_guo 3DVAR 7 16 serial|dmpar 6 cv3_guo_32 3DVAR 9 16 serial|dmpar 7 cwb_ascii 3DVAR 4 16 serial|dmpar 8 cwb_ascii_outerloop_rizvi 3DVAR 4 16 serial|dmpar 9 cwb_ascii_thinning 3DVAR 4 16 serial|dmpar 10 iasi_kavulich 3DVAR 16 16 dmpar #11 lat_lon_outerloops 3DVAR 16 16 dmpar 12 outerloop_bench_guo 3DVAR 16 16 serial|dmpar 13 outerloop_ztd_bench_guo 3DVAR 1 16 serial|dmpar 14 radar_meixu 3DVAR 8 16 serial|dmpar 15 radar_meixu_thinning 3DVAR 4 16 serial|dmpar 16 rainfall_outerloops 4DVAR 16 16 dmpar 17 rttov_4dvar_madagascar 4DVAR 32 16 dmpar 18 seviri_varbc VARBC|3DVAR 16 16 dmpar 19 sfc_assi_2_outerloop_guo 3DVAR 16 16 serial|dmpar 20 t44_liuz 3DVAR 2 16 serial|dmpar 21 t44_prepbufr 3DVAR 8 16 serial|dmpar 22 tut_xinzhang_4dvar 4DVAR 32 16 dmpar 23 tut_xinzhang_fgat FGAT 16 16 dmpar 24 tut_xinzhang_obsproc OBSPROC|3DVAR 16 16 dmpar 25 tut_xinzhang_rttov_genbe GENBE|3DVAR 10 16 dmpar ... The main script reads this file to find the parameters of each test, such as test name, parallelism, number of cores, test type (3DVAR, VARBC, etc.). Other important data such as project number, Yellowstone queue, and locations of the databases are specified here as well There is a separate section for each compiler/platform combination To run only specific tests, you can simply comment out a line to omit that test

What the test database looks like [kavulich@yslogin4 REGTEST]$ ls WRFDA-data-EM afwa_t7_ssmi cwb_ascii_outerloop_rizvi radar_meixu_thinning t44_prepbufr afwa_t7_ssmi_32 cwb_ascii_thinning rainfall_outerloops tut_xinzhang_4dvar ASR_airs iasi_kavulich rttov_4dvar_madagascar tut_xinzhang_fgat ASR_prepbufr lat_lon_outerloops seviri_varbc tut_xinzhang_obsproc cv3_guo outerloop_bench_guo sfc_assi_2_outerloop_guo tut_xinzhang_rttov_genbe cv3_guo_32 outerloop_ztd_bench_guo share wind_sd cwb_ascii radar_meixu t44_liuz Each of those is a directory containing the files needed as input for the section of WRFDA being tested. For 3DVAR tests this can be as simple as a first guess, be.dat, and namelist.input, but for others there can be many more

What the baseline database looks like [kavulich@yslogin4 REGTEST]$ ls BASELINE.NEW wrfvar_output.Linux.yellowstone.afwa_t7_ssmi_32.dmpar.gfortran wrfvar_output.Linux.yellowstone.afwa_t7_ssmi_32.dmpar.ifort wrfvar_output.Linux.yellowstone.afwa_t7_ssmi_32.dmpar.pgi wrfvar_output.Linux.yellowstone.afwa_t7_ssmi_32.serial.gfortran wrfvar_output.Linux.yellowstone.afwa_t7_ssmi_32.serial.ifort wrfvar_output.Linux.yellowstone.afwa_t7_ssmi_32.serial.pgi wrfvar_output.Linux.yellowstone.afwa_t7_ssmi.dmpar.gfortran wrfvar_output.Linux.yellowstone.afwa_t7_ssmi.dmpar.ifort wrfvar_output.Linux.yellowstone.afwa_t7_ssmi.dmpar.pgi wrfvar_output.Linux.yellowstone.afwa_t7_ssmi.serial.gfortran wrfvar_output.Linux.yellowstone.afwa_t7_ssmi.serial.ifort wrfvar_output.Linux.yellowstone.afwa_t7_ssmi.serial.pgi wrfvar_output.Linux.yellowstone.ASR_airs.dmpar.gfortran wrfvar_output.Linux.yellowstone.ASR_airs.dmpar.ifort wrfvar_output.Linux.yellowstone.ASR_airs.dmpar.pgi ... Each of those is a wrfvar_output file to be compared. There is one baseline file for each test, parallelism, and compiler combination.

What the test looks like Experiment Paropt Job type CPU_MPI Status Walltime(s) Compare ================================================================================================= ASR_airs dmpar 3DVAR 16 running 0 -- ASR_prepbufr dmpar 3DVAR 16 running 0 -- ASR_prepbufr serial 3DVAR 16 pending 0 -- cwb_ascii dmpar 3DVAR 4 done 27 match cwb_ascii serial 3DVAR 4 running 0 -- cwb_ascii_outerloop_rizvi dmpar 3DVAR 4 running 0 -- cwb_ascii_outerloop_rizvi serial 3DVAR 4 pending 0 -- cwb_ascii_thinning dmpar 3DVAR 4 done 32 match cwb_ascii_thinning serial 3DVAR 4 pending 0 -- iasi_kavulich dmpar 3DVAR 16 running 0 -- lat_lon_outerloops dmpar 3DVAR 16 error 6 Output missing outerloop_ztd_bench_guo dmpar 3DVAR 1 done 14 match outerloop_ztd_bench_guo serial 3DVAR 1 done 9 match radar_meixu dmpar 3DVAR 8 done 31 match radar_meixu serial 3DVAR 8 pending 0 -- rainfall_outerloops dmpar 4DVAR 16 running 0 -- rttov_4dvar_madagascar dmpar 4DVAR 64 running 0 -- seviri_varbc dmpar VARBC|3DVAR 16 running 0 -- sfc_assi_2_outerloop_guo dmpar 3DVAR 16 running 0 -- sfc_assi_2_outerloop_guo serial 3DVAR 16 pending 0 -- t44_liuz dmpar 3DVAR 2 done 18 match t44_liuz serial 3DVAR 2 running 0 -- tut_xinzhang_4dvar dmpar 4DVAR 32 running 0 -- tut_xinzhang_fgat dmpar FGAT 16 running 0 -- tut_xinzhang_obsproc dmpar OBSPROC|3DVAR 16 running 0 -- tut_xinzhang_rttov_genbe dmpar GENBE|3DVAR 10 running 0 -- wind_sd dmpar 3DVAR 16 running 0 -- wind_sd serial 3DVAR 16 pending 0 -- dmpar job for outerloop_bench_guo was completed in 41 seconds. Comparing 'outerloop_bench_guo/wrfvar_output.Linux.yellowstone.outerloop_bench_guo.dmpar.gfortran' to '/glade/p/work/kavulich/REGTEST/BASELINE.NEW/wrfvar_output.Linux.yellowstone.outerloop_bench_guo.dmpar.gfortran'

When the test is complete The full test takes about 20 minutes to complete (8-12 minutes compilation, ~5 minutes for running tests) The compiled code was compiled separately for 3DVAR and 4DVAR, as well as a separate directory for each parallelism option: WRFDA_3DVAR_serial/ WRFDA_3DVAR_dmpar/ WRFDA_4DVAR_dmpar/ If you chose the option “--upload=yes”, or the source of the code is the repository, the test summary will be uploaded to the website At the conclusion of the test, you will receive an email with the test results

Script options There are many options which can be specified by providing arguments to the main script --compiler=COMPILER This one is required --source=SOURCE_CODE.tar This is the location of the source code. Can also specify “--source=SVN” to check out the code from the main WRF repository. --revision=#### If you use the “SVN” option, you can specify a specific revision number to test --upload=yes Uploads a test summary to the website upon completion --exec=yes Instead of compiling the code, use pre-compiled code. Make sure the directory names are correct! --j=## For parallel compilation, specify the number of processors (default is 4)

Other notes The source code tar file must contain a directory named “WRFDA” that contains the source code The test removes directories that are removed for the release: chem, hydro, and dyn_nmm All this information and more can be found in the README file found in the test repository and on the website