C-IFS: How are developments integrated Anna Agusti Panareda, Samuel Remy, Vincent Huijnen, J.J-Morcrette, Olaf Stein, Joaquim Arteta, Simon Chabrillat, Johannes Flemming & Angela Benedetti, Antje Inness, Sebastien Massart, Richard Engelen as well as all contributors to IFS and C-IFS

IFS : Integrated Forecasting System of ECMWF A very good NWP forecast and data assimilation model

IFS : Integrated Forecasting System of ECMWF A complex model system for forecast and assimilation 10th anniversary of IFS 1997

Adding composition to IFS : Composition -IFS In GEMS project: Coupled system IFS-MOZART for chemistry GHG and aerosol on-line (integrated) in the IFS MACC I-III: chemistry on-line in IFS Chemistry - IFS (2009) Renamed to Composition –IFS: all composition aspects Composition – IFS : global production system in CAMS at ECMWF

Integration of chemistry & aerosol modules in IFS C-IFS On-line Integration Coupled System IFS- MOZART3 / TM5 Fast, consistent but higher coding effort Flexible but very un-efficient Flemming et al. 2009

Composition – IFS : multiple schemes MOZART chemistry Cariolle Strat. O3 CO2 & CH4 GLOMAP aerosol MOCAGE chemistry CAMS Procurement Open IFS Interface BASCOE stratospheric chemistry TM5 (CB05) MACC (LMDz) aerosol BMS MACC III heritage

Benefits for CAMS using C-IFS IFS is the best NWP model on the planet IFS is a very efficient global model Operational IFS resolution is currently 9 km globally CAMS o-suite resolution is 40 km globally IFS data assimilation (4D-VAR, ENS) used for composition Using 4D-Var algorithm (Ensemmble DA) Infra structure to process assimilated observations

Benefits of high resolution model Mid-tropospheric CH4 [ppb] at 450 hPa Low resolution FC (80 km, L60) High resolution FC (16 km, L137) Anna Agusti-Panareda

Challenges to use C- IFS for CAMS Adaptation of data assimilation system to specifics of composition field and observations IFS advection does not formally conserve mass Global mass fixers implemented Link CAMS development with ongoing IFS development 2-3 new cycles each year Reproducibility of older cycles IFS coding standards

Towards better integration between C-IFS Components Between Chemistry, Aerosols and GHG modules Secondary aerosol formation based on chemistry Photolysis and surface chemistry modulation by aerosol Unified modelling of methane in Chemistry and GHG Code harmonisation Composition on NWP (and back !!) Aerosol in radiation Ozone in radiation Land surface and fluxes (emissions and deposition)

CAMS ozone fields in IFS radiation scheme I

CAMS ozone fields in IFS radiation scheme II New CAMS Ozone climatology used in next IFS cycle

How are C-IFS developments by CAMS partners integrated …

IFS - coding rules http://intra. ecmwf REAL(KIND=JPRB),INTENT(IN) :: PRR(KLON,NREAC) REAL(KIND=JPRB),INTENT(IN) :: PRJ(KLON,NPHOTO) DO JL=KIDIA,KFDIA ZP1=PRJ(JL,jbno3)*PY(JL,ino3) …… ENDDO not a coding rule but advised for efficiency

Code efficiency Use Profiling to find bottle necks Usage of resources per routine call Profiling information for program='/fws2/lb/work/rd/disr/g99u/2014120100/gfc/tmp.g99u_fc_fcgroup1.model.1.32453/ifsMASTER', proc#3: No. of instrumented routines called : 1254 Instrumentation started : 20160502 143312 Instrumentation ended : 20160503 035310 Instrumentation overhead: 35.69% Memory usage : 1449 MBytes (heap), 1452 MBytes (rss), 0 MBytes (stack), 0 (paging) Total CPU-time is 98250.51 sec on proc#3, 0 MFlops (ops#0*10^6), 0 MIPS (ops#0*10^6) (32 procs, 2 threads) Thread#1: 55730.71 sec (56.72%), 0 MFlops (ops#0*10^6), 0 MIPS (ops#0*10^6) Thread#2: 42519.80 sec (43.28%), 0 MFlops (ops#0*10^6), 0 MIPS (ops#0*10^6) # % Time Cumul Self Total # of calls MIPS MFlops Div-% Routine@<thread-id> (Size; Size/sec; Size/call; MinSize; MaxSize) (self) (sec) (sec) (sec) 1 13.13 12900.110 12900.110 12900.290 563 0 0 0.0 >MPL-TRGTOL_COMMS ( 803)@1 2 8.53 21278.660 8378.550 16104.600 56683200 0 0 0.0 *UKCA_DCOFF_PAR_AV_K@1 3 8.41 29536.700 8258.040 15986.450 55751976 0 0 0.0 *UKCA_VGRAV_AV_K@2 4 8.37 37761.020 8224.320 16033.160 56683200 0 0 0.0 UKCA_VGRAV_AV_K@1 5 8.33 45946.390 8185.370 15861.140 55751976 0 0 0.0 UKCA_DCOFF_AR_AV_K@2 6 4.73 50594.410 4648.020 37276.250 4048800 0 0 0.0 *UKCA_DDEPAER_INCL_SEDI@1 7 4.58 55095.040 4500.630 36813.330 3982284 0 0 0.0 UKCA_DDEPAER_INCL_SEDI@2 S.Remy, C-IFS GLOMAP profiling

How are C-IFS developments integrated ? RD research dep. FD forecast dep. IFS team RD ECMWF CAMS partner CAMS ECMWF CAMS FD ECMWF Up to 1 year from development to o-suite implementation e-suite CAMS VAL o-suite

Thank you! ευχαριστώ Tower of Winds A meteorological monument nearby with a CAMS theme: Skiron (NW) distributes the ashes

How are C-IFS developments integrated … Contributing partner (or ECMWF): Testing (Test A) of individual model development Delivery to ECMWF/CAMS CAMS-ECMWF Section: Integrate development in CAMS branch Testing (Test B) of all integrated model improvements Submit to ECMWF RD IFS section for ECMWF cycle upgrade ECMWF RD IFS group Merge new cycle from all ECMWF contributions Forecast Department Copernicus section: Run experimental CAMS suite (e-suite) and tested by VAL Run operational CAMS suite (o-suite) Each of the steps can take 1-3 month so that it takes up to a year month from model update to implementation in o-suite Time line of ECMWF cycle upgrades will be announced to CAMS partners well in advance

Computational Cost C-IFS 80 km 40 km 16 km