7/9/20161 cms.ncas.ac.uk March 2014 NCAS Unified Model Introduction Part 1: Overview of the UM system York – March 2014

March What is the UM? UM User Interface (UMUI) Running the UM Contents

March The Unified Model is…  owned and developed by UK Met Office, used for both commercial weather forecasts as well as climate research and advice to government, for example IPCC reports  PUM (Ported UM) is used under licence by UK academic community for weather and climate research  used under licence by forecast/climate agencies around the world [Norway, South Africa, Australia…]  The UM is used by a large and diverse community of users.  The UM is expected to run on a diverse range of computers.

March At the UK Met Office Unified Model code Atmosphere Ocean Data assimilation system UKMO Data archive Observations UMUI SCSCAP

March External UM Unified Model code Atmosphere Ocean UMUI Contains 1)UM code + fixes 2)UM User guide + documentation + code browser 3)UM Tools 4)Installation examples + 2 or 3 example jobs tested on 2 or 3 computers+compilers

March NCAS supported UM on HPC services (HECToR, MONSooN) Unified Model code Atmosphere Ocean UMUI Contains 1)UM code installed, optimised, tested and fixed 2)Changes to UMUI and archiving mechanism 3)Tools and utilities both from UKMO and NCAS 4)Managed data from Met Office and ECMWF + many user jobs, both climate and NWP HPC Data archive

March UM Versions The Met Office (UKMO) release new UM versions regularly (approx every 3 months). Each version is independent and stands alone. NCAS may not always port each new version to HECToR or MONSooN NCAS has very limited effort to port new versions to external systems NCAS carries out limited testing and provides only sample jobs for users Old Dynamics New Dynamics Rose UKMO x NCAS: ARCHER (N)y Y Y Y…………………………. MONSooN Y Y Y Y…………………………. Code Maintenance nupdate pumscm FCM Ocean Code old old old NEMO

March Unified Model (UM) versions configurations Hadcm3 Hadgam1 Hadgem3-AO experiments Nugam IPCC scenario A3  The UM is not a single fixed piece of code.  In the UK academic community there is a broad UM spectrum as multiple UM versions are used new UM configurations are developed a large variety of UM experiments are performed QESM PS Hadgem2-ES 7.6

March The Unified Model (UM) is a collection of numerical modelling software which … Can be used in atmosphere only, ocean only, or coupled or with a slab ocean Can be coupled to other ocean models via OASIS e.g. Hadopa, or other components NEMO, CICE, JULES (for example in HadGEM3) May be run in many different modes  aquaplanet  global or mesoscale  regional coupling

March runs at different resolutions  Global climate (N48, N96, N144, N216, N512)  LAM (Limited Area Model) (UK 12 km, 4 km)  L38 - L60 – increased vertical resolution  Higem – N144 atmosphere + 1/3 ⁰ ocean N48N96N144N km135 km90 km60 km 96 x x x x º x 2.4º1.88º x 1.24º1.25º x 0.83º0.83º x 0.56º

March What is the UM? UM User Interface (UMUI) Running the UM

March UMUI – UM User Interface The UMUI… is a windows interface to the UM is written in Tcl/Tk is currently offered via the PUMA service – which provides all the latest UMUI changes – centralised access to local files (hand-edits etc) – common job database for ALL academic users creates the scripts which control the UM run (some changes to these scripts may be needed for new bits of code – hand-edits) has been extended to accommodate non-UKMO users assigns values to namelist switches used by the UM at runtime selects code branches used to build the model (UM6.6.3+)

March UMUI – creating UM jobs Choose a UM job which is close to the one you want to run either from a colleague, UKMO or from NCAS - either copy the job into your own job if they are in the same local database - or upload a basis file, the full description of a UM job which has come from another UMUI database, into your own job NCAS standard jobs should be available under the userid umui Basis files can be downloaded from the UMUI and ed.

March UMUI - vocabulary Experiments : xxab - a grouping of UM jobs - identified by a description in the UMUI - only your experiments are displayed by the UMUI, unless otherwise requested - experiments can be created, copied and deleted Jobs : xxabc - up to 26 jobs can be grouped in an experiment - jobs can be created, copied and deleted - 2 jobs can be differenced Navigation windows input windows

March

March UM – Submitting the model UMUI scripts namelists code MONSooN HECToR submit What do you need to know about the system where the UM is installed? directory structure and disk space setup job submission mechanism and queue structures input files available e.g. branches, start files, ancillary files The UM uses namelists for setting parameters at runtime, these are set in the UMUI.

March What is the UM? UM User Interface (UMUI) Running the UM

March Running a UM job: Stage 1 - Compilation UMUI – what options? Compilation and Run Options -> Compile and run options for Atmos & Recon Options : compile only, compile and run, run from a named executable Compile Overrides: Used on some computers for compiler options to speed up some sections of the code or to overcome problems. FCM manages code components creates makefiles for compilation and loading compiles and links the code according to options selected creates an executable

March Running a UM job: Stage 2 - Reconfiguration Reconfiguration is a standalone program which modifies (reconfigures) UM atmosphere or ocean start files FCM code management compiles the code runs the reconfiguration in parallel for new dynamics in the serial queue for the old dynamics produces new start files UMUI – what options? Compilation and Run Options -> Compile and run options for Atmos & Recon Options: run from an existing executable, or compile and create an executable

March Running a UM job: Stage 3 - Running UMUI – what options? Input/Output Control & Resources -> Start date and run length options For length of model run and the number of processors to use you need to understand the computer’s queue structure and the performance of your job. User Information & Submit Method -> Job submission method qsub for Cray XE6 LoadLeveller for for PCs and workstations

March STASH Output Filenames Job Output UM Output

March STASH Spatial and Temporal Averaging and Storage Handling 1) Select diagnostic from Load New Diagnostic section 2) Time profilewhen diagnostic will be output (start and end time and frequency) whether time processing required (accumulation, mean, time series) 3) Domain profile vertical (specify which levels) horizontal (limited area, meaning [ zonal, vertical, meridional, horizontal], weighting) 4) Usage profile select output unit for the diagnostic

March

March STASH in the UMUI The UMUI windows for STASH are different from model windows. Available diagnostics are organised in sections. Some diagnostics have HELP information, a lot don’t! Diagnostics -> load new diagnostics -> (double click on the section) Just because a diagnostic is available doesn’t mean it works!

March Verify diagnostics (Ctrl+V) Package switched onPackage switched off

March UM - STASHmaster file H1| SUBMODEL_NUMBER=1 H2| SUBMODEL_NAME=ATMOS H3| UM_VERSION=8.4 # #|Model |Sectn | Item |Name | #|Space |Point | Time | Grid |LevelT|LevelF|LevelL|PseudT|PseudF|PseudL|LevCom| #| Option Codes | Version Mask | Halo | #|DataT |DumpP | PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8 PC9 PCA | #|Rotate| PPF | USER | LBVC | BLEV | TLEV |RBLEVV| CFLL | CFFF | # #=============================================================================== # 1| 1 | 0 | 2 |U COMPNT OF WIND AFTER TIMESTEP | 2| 2 | 0 | 1 | 18 | 1 | 1 | 2 | 0 | 0 | 0 | 0 | 3| | | 1 | 4| 1 | 2 | | 5| 0 | 56 | 0 | 65 | 0 | 0 | 0 | 0 | 5 | # Contains all the information on all the atmosphere diagnostics available

March STASH - Adding new diagnostics Users can add new diagnostics Advice via the CMS website (STASH User Guide) User must provide a STASHmaster file local to the UMUI Users can copy diagnostics settings from one UM job to another using: copy_stash This script takes as input the two UM basis files. The two jobs must be the same UM version.

March 2014 Output listing from the UM 28 Output listing quite large and confusing Output directory $HOME/um/umui_out Output listing filename, e.g. xagmc000.xagmc.o98342.t14136.leave Output listing options controlled by UMUI Input/Output Control & Resources -> Output Choices Check output listing for - timings reported at the end of the listing file - key words like ERROR, ABORT, "file not found" Check presence of key files, executable after compilation start file after reconfiguration output files after model run

March SUMMARY

March How a UM experiment (HADAM3, UKMES, UK4km) is defined. A standard named UM experiment has a defined version of the UM (4.5, 6.1) a particular horizontal and vertical resolution particular settings in the UMUI + hand-edits, which change the settings in the UMUI include a controlled list of ‘mods’, which are changes to the UM source code or particular branches in FCM use a defined set of input files, start files, ancillary files and lateral boundary conditions => Named experiments can change and evolve at a different rate to UM versions

March 2014 NCAS Unified Model Introduction Part 2: Code Maintenance

March … is different for different versions Version 4.5 nupdate (either original Cray version or UKMO C version) Version 6.1 pumscm (almost the same as nupdate written in perl) Version 6.3 and above FCM (Flexible Configuration Management) - written in perl - based on subversion UM Code Maintenance…

March 2014 UM 4.5 and UM User Interface (UMUI) PUMA UM code library Input files Output Analysis Analysis and Visualisation Local 1) Setting up the UM job 2) Running the UM job 3) Analysis of UM Output The UMUI produces a set of scripts to run the UM that assumes that the UM has been installed with the structure created by the installation procedure. Super Computer

March UM 4.5/6.1 - Code UM code is written in Fortran with some C Fortran 77 with Fortran 90 extensions as well as Fortran 90 uses NUPDATE (and derivatives) as a code maintenance tool - either former Cray version now ported to IBM, Linux, HP, DEC, Sun, SGI - or UMKO portable version with limited functionality - or PUMSCM (in perl) for the New Dynamics versions NUPDATE user guide available on the NCAS-CMS web site, PUMSCM has nearly the same functionality but with some extra features.

March UM 4.5/6.1 - NUPDATE/PUMSCM DECK FRED DECK JOE Use a modification set (modset or mod) to change the program library. *IDENT TESTP */ test print statement *DECLARE FRED *I FRED.24 print *, ‘ temp in CONVEC’, T Program Library each line in the program library has a unique identifier. a new program library is only created for new versions of the UM

March UM 4.5/6.1 - NUPDATE/PUMSCM Program Library (for each version of the UM) modset (change or modification of the code) nupdate or pumscm *.f (compile file for each subroutine or deck) defs (nupdate controlled definitions) apply

March From UM onwards changes to the Unified Model source code is managed using: FCM (Flexible Configuration Management) source code managed through subversion full history of source code change using trac and wiki code extract system managed through scripts build system with scripts and make UM onwards - Code

March UMUI UM code Input files Output Analysis Analysis and Visualisation Local subversion +trac+wiki User logs in User UM code HPC service PUMA NCAS UM FCM service is only available on PUMA. UM6.6.3 onwards