2. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Introduction to the Standard Compile Environment (SCE) of the Integrated Model & Data Infrastructure (IMDI) Stephanie Legutke, MPI-M M&D

3. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Background PRISM (Dec 2001-Nov 2004): Develop for the European Earth System (ES) modelling community a software infrastructure to  set up  compile  run  analyse coupled ES model experiments and thereby establish a European ES research network M&D: Make the SCE and SRE developed in PRISM become part of the IMDI

4. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Requirements... … to the SCE infrastructure software  Integrate any ES climate research model …  … on any platform  Allow for easy replacement of components in coupled models (CplM)  Interface to a GUI

5. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Requirements …  Minimum impact on component source code  Give a common look&feel with all models and experiments  Automate but allow for easy customizing  Keep it simple  [Low maintenance] … from the model developers and users

6. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Components of the SCE  Coupled models, component main & sub models, and libraries  Standards  High level design of the compile system  Compile time configuration of coupled/component models  Component model compile scripts generation  (g)make (file)

7. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke OCEAN Coupled Earth System Models Sea Ice ATMOSPHER E Marine Biogeo- chemistry Land Surface Atmospheric Chemistry OASIS3

8. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Coupled Earth System Models OCEAN ATMOSPHER E Marine Biogeo- chemistry Land Surface Atmospheric Chemistry Another Sea Ice OASIS3 Main models Sub models Message passing SBR parameter list

9. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke SCE standards for....  Package rule for main&sub model exchange: No use of other component‘s data blocks ; use message passing or SBR param. lists instead  OASIS3 software if different executables: via OASIS executable or directly using PSMILe(MPI) calls (PSMILe=PRISM System ModelI Interface Library)  Configuration for coupling with partner models by conditional compiling (cpp flag ‘__cpl_partnermodel‘)... coupling

10. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke SCE standards for....... component model codes  Components have to be independent from each other (for flexible component exchange)  Avoid unresolved references by cpp flag wraping  Standard directory structure (to enable use of simple tools)... source code  Single F90 MODULE per file  File name = MODULE name  Suffixes: F90, f90, F, f, c  Single occurence of each basename  Non portable code wraped with cpp flags

11. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke USE, CALL, INCLUDE SCE: Source code directory ok ok for call of SBR if mod2 is submodel of mod1

12. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke SCE: Source code reorganisation before after

13. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke COMP_mod1_ID.node User lib*.a call COMP_libs.node ”list2” return Model+submodel compilation mod1_mod2_ID[.MPIn].x MPIn=MPI or MPI2 COMP_libs.node ”list1” call mod2 / libmod2_ID[.MPIn].a COMP_mod2_ID.node call return mod1 / libmod1_mod2 _ID [.MPIn].a

14. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Build directory (libraries and executables)

15. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke COMP_mod1_ID.node Component model compilation mod1_mod2_ID[.MPIn].x lib*.a mod2 / libmod2_ID[.MPIn].a mod1 / libmod1_mod2 _ID [.MPIn].a lib*.a mod2 / libmod2_ID[.MPIn].a mod1 / libmod1_mod2 _ID [.MPIn].a COMP_libs.node ”list1” COMP_mod2_ID.node COMP_libs.node ”list2” call return User MPIn=MPI or MPI2

16. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke COMP_libs.node “list” COMP_mod1_ID.node all / lib / clean / tar “list” User list of libraries: make src / Makefile *.o / *.mod Model compilation Top_Makefile make create (in-doc.) mod1_ID[_MPIn].x src directories: make Makefile lib*.a Top_Makefile make create (in-doc.)

17. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Build directory: *.o/*.mod *.o *.mod *.a modn *_ID.a *_ID.x *.o *.mod

18. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Utilities & Tools for compilation  … for the specific platform  … for the specific component model  … configured for the coupled configuration No compile scripts are provided … … but tools that generate them … … Example how to do that =>

19. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Creating compile scripts Create_COMP_cpl_mod.ksh cplmod [ID [node]] Create_COMP_models.frm mod1 MPI[1,2] - - node ID “mod1 mod2 mod3” Create_COMP_models.frm mod2 MPI[1,2] - - node ID “mod1 mod2 mod3” Create_COMP_models.frm mod3 MPI[1,2] - - node ID “mod1 mod2 mod3” Create_COMP_models.frm oasis3 MPI[1,2] - - node Create_COMP_libs.frm MPI[1,2] - - node COMP_mod1_ID.node COMP_mod2_ID.node COMP_mod3_ID.node COMP_oasis3.node COMP_libs.node mod1_mod2_ID.MPI[1,2].x mod2_ID.MPI[1,2].a mod3_ID.MPI[1,2].x oasis3.MPI[1,2].x …., psmile.MPI[1,2].a cplmod = mod1+mod2 & mod3 & OASIS3: bold: to be typed by user (scripting) additional configuring by editing

20. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke … include GUIspecific_$model.h include Sitespecific_$node.h include Cppflags_$model.h include Compile_mode_$model_$node.h include Make_model.h … m4 input file Compile Script Assembly Create_COMP_models.frm model “” – “” node ID “ partner models ” COMP_model_ID.node m4 *.h

21. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke list of libraries: make src / Makefile COMP_libs.node “list” call with libraries “list” SCE : Model compilation Top_Makefile make create (in-doc.) mod1_ID[_MPIn].x src directories: make Makefile lib*.a Top_Makefile make create (in-doc.) *.o / *.mod COMP_mod1_ID.node all / lib / clean / tar User

22. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke SCE: (GNU) Make All compilation is based on the (g)make software: Make ‘targets’ having ‘prerequisites’ with well defined ‘rules’ while avoiding redundant actions: Make –f Makefile … Target: prerequisite1, prerequisite2 …. rule … Targets, rules & prerequisites are defined in Makefile :

23. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke (GNU) Make targets  Executables: OASIS3.x, ECHAM5.x, *.x  Libraries: *.a  Model libraries: *_ID.a  Binary object code: *.o  FORTRAN90 Modules: *.mod

24. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke (GNU) Portable Makefiles  All non portable code exported from the compile scripts  1 Makefile for each source directory with compiler input  All CMs Makefiles have the same look and feel  Exception: prerequisites Tool for generation of *.o prerequisites  All library Makefiles have the same look and feel

25. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Example Makefile: Def. of variables SRCS1 = $(shell find../../../../src/mod/$(MODEL_DIR)/$(strip $(SRC)) –name '*.F90' -print) OBJS1 = $(patsubst../../../../src/mod/$(MODEL_DIR)/$(strip $(SRC))/%.F90, %.o, $(SRCS1)) …. SRCS5 = $(shell find../../../../src/mod/$(MODEL_DIR)/$(strip $(SRC)) -name '*.c' -print) OBJS5 = $(patsubst../../../../src/mod/$(MODEL_DIR)/$(strip $(SRC))/%.c, %.o, $(SRCS5)) VPATH =.:../../../../src/mod/$(MODEL_DIR)/$(strip $(SRC)):\../../../../src/mod/$(MODEL_DIR)/$(strip $(SRC))/include:\../../../../src/mod/$(MODEL_DIR)/include:\../../lib/psmile.${CHAN}:\ $(NETCDF_INCLUDE):$(MPI_INCLUDE):$(SYS_INCLUDE):\../../../lib/$(MODEL_DIR):\../../../lib/$(SUBMOD1):\../../../lib:\../../../../src/mod/$(MODEL_DIR)/make_dummies LIBRARY =../../../lib/$(MODEL_DIR)/lib$(MODLIB).a PROG =../../../bin/$(EXEC).x INCLS = -I../../../../src/mod/$(MODEL_DIR)/include \ -${I4mods}../../lib/clim.${CHAN} INCLSC = -I../../../../src/mod/$(MODEL_DIR)/include

26. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Example Makefile: Targets & Rules clean: rm -f $(LIBRARY) i.* *.o *.mod Top_Makefile_* all: $(PROG) lib: $(LIBRARY) $(PROG): $(DEPLIBS) $(LIBRARY) $(F90) $(LDFLAGS) -o $@ $(MAINPRG).o $(LIBS) $(LIBRARY): $(OBJS1) $(OBJS2) $(OBJS3) $(OBJS4) $(OBJS5) $(AR) $(ARFLAGS) $(LIBRARY) *.o.SUFFIXES:.SUFFIXES:.o.c.f.F.f90.F90 %.o: %.F90 $(F90) $(F90FLAGS) $(INCLS) -c $< … %.o: %.f $(f) $(fFLAGS) $(INCLS) -c $< %.o: %.c $(CC) $(CCFLAGS) $(INCLSC) -c $<

27. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Generating.o prerequisites mod_unitncdf.o: mod_kinds_oasis.o alloc_src.o: mod_anais.o mod_analysis.o mod_coast.o mod_experiment.o \ mod_extrapol.o mod_kinds_oasis.o mod_memory.o mod_nproc.o \ mod_parallel.o mod_parameter.o mod_pipe.o mod_rainbow.o mod_sipc.o \ mod_string.o mod_timestep.o mod_unitncdf.o dealloc_src.o: mod_anais.o mod_analysis.o mod_coast.o mod_experiment.o \ mod_extrapol.o mod_kinds_oasis.o mod_memory.o mod_nproc.o \ mod_parallel.o mod_parameter.o mod_pipe.o mod_rainbow.o mod_sipc.o \ mod_string.o mod_timestep.o mod_unitncdf.o extrap.o: mod_extrapol.o mod_kinds_oasis.o mod_parameter.o mod_printing.o \ mod_unit.o getfld.o: mod_analysis.o mod_clim.o mod_experiment.o mod_hardware.o \ mod_kinds_oasis.o mod_label.o mod_memory.o mod_parameter.o \ mod_printing.o mod_sipc.o mod_string.o mod_timestep.o mod_unit.o \ mod_unitncdf.o netcdf.inc Append_Dependencies model_name src_directory

28. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke SCE : Util directory Structure

29. 22-24 Feb 05 Workshop DKRZ / M&D Stephanie Legutke Documentation & URLs  Handbook The PRISM Standard Compilation Environment Stephanie Legutke and Veronika Gayler PRISM Report Series, No. 4, 1. Edition, 15 Jan 2005 http://prism.enes.org/Results/Documents/PRISMReports/Report04.pdf http://prism.enes.org/Results/Documents/PRISMReports/Report04.pdf  WP3i (Model Assembly) M&D PRISM web site http://prism.dkrz.de/Workpackages/WP3i/Documentations http://prism.dkrz.de/Workpackages/WP3i/Presentations http://prism.dkrz.de/Workpackages/WP3i/Documentations http://prism.dkrz.de/Workpackages/WP3i/Presentations  PRISM project web site http://prism.enes.org http://prism.enes.org  PRISM Portal (under development) http://prism.enes.org/Portal http://prism.enes.org/Portal