1. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 How to work with the user-interface-routine in PALM Contents: Purpose of the user-interface Embedding in PALM Realisation Hints for new users

2. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 Purpose of the user-interface a standardarized model cannot count for all specific interests => the user should be able to add special, non-standard features of the model without changing the standard code for example –heating by increasing the surface temperature pt(nzb,:,:) = pt_surface + simulated_time * constant –inhomogeneous surface pt(nzb,:,x1:x2) = pt_ice pt(nzb,:,x2:x3) = pt_water pt(nzb,:,x3:x4) = pt_ice z0(:,x1:x2) = roughness_ice z0(:,x2:x3) = roughness_water z0(:,x3:x4) = roughness_ice t waterice windheat flux every timestep at begin user namelist parameters

3. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 Embedding in PALM …/SOURCE/user_interface.f90 recommendations: 1.create directory: …/SOURCE/USER_CODE/ example 3.cp …/SOURCE/user_interface.f90 …/SOURCE/USER_CODE/ example/user_example.f90 4. add your own user-code and use mrun with the option -s TEST (ensure that this file has write permits!) name of the run excerpt from.mrun.config... %add_source_path $HOME/palm/current_version/SOURCE/USER_CODE/$fname...

4. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 subroutines in user_interface user_last_actions Initialization: Run-time: Finishing: user_parin user_header user_init user_actions user_statistics user_init_particles user_particles_attributes user_dvrp_coltab example "inhomogeneous surface" example "heating by increasing" user_particles_attributes user_dvrp_coltab

5. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 Initialization

6. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 statistic regions default times series and vertikal profiles are averaged horizontally over the whole model domain with defining statistic regions in user_init averaging will be made for this regions: rmask(nys-1:nyn+1,nxl-1:nxr+1,0:9) = rmask is defined in the module statistic and must be set in user_init 0 belongs not to region 1 belongs number of the region

7. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 Run-time standard advection

8. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 Realisation: location user_actions Excerpt from time_intergration.f90 ! !-- Start of the time loop DO WHILE ( simulated_time < end_time.AND..NOT. stop_dt.AND. &.NOT. terminate_run )... ! !-- Execute the user-defined actions #if defined( __decalpha ) PRINT*, '+++ WARNING: leap_frog: no call of user_actions due to' PRINT*, ' internal compiler error' #else CALL user_actions( 'before_timestep' ) #endif... ! !-- Solve the prognostic equations. A fast cache optimized version with !-- only one single loop is used in case of Piascek-Williams advection !-- scheme. NEC vector machines use a different version, because !-- in the other versions a good vectorization is prohibited due to !-- inlining problems. IF ( host(1:3) == 'nec' ) THEN CALL prognostic_equations_vec ELSE IF ( impulse_advec == 'ups-scheme'.OR. & scalar_advec == 'ups-scheme'.OR. & scalar_advec == 'bc-scheme' ) & THEN CALL prognostic_equations ELSE CALL prognostic_equations_fast ENDIF ….. ! !-- Execute user-defined actions #if defined( __decalpha ) PRINT*, '+++ WARNING: leap_frog: no call of user_actions due to' PRINT*, ' internal compiler error' #else CALL user_actions( 'after_timestep' ) #endif

9. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 Realisation: location user_actions Excerpts from prognostic_equations.f90 … ! !-- Tendency terms for u-velocity component tend(:,j,i) = 0.0 IF ( bt == 2.0 ) THEN CALL advec_u_pw( i, j ) CALL diffusion_u( i, j, ddzu, ddzw, km_m, tend, u_m, usws_m, v_m, & w_m ) ELSE CALL advec_u_up( i, j ) CALL diffusion_u( i, j, ddzu, ddzw, km, tend, u, usws, v, w ) ENDIF CALL coriolis( i, j, 1 ) IF ( sloping_surface ) CALL buoyancy( i, j, pt, 1, 4 ) CALL user_actions( i, j, 'u-tendency' ) ! !-- Prognostic equation for u-velocity component DO k = nzb+1, nzt u_p(k,j,i) = ( 1-at ) * u_m(k,j,i) + at * u(k,j,i) + & bt * dt_3d * tend(k,j,i) - & dt_3d * ( p(k,j,i) - p(k,j,i-1) ) * ddx - & bt * rdf(k) * ( u(k,j,i) - ug ) ENDDO ! !-- Tendency terms for v-velocity component tend(:,j,i) = 0.0 IF ( bt == 2.0 ) THEN CALL advec_v_pw( i, j ) CALL diffusion_v( i, j, ddzu, ddzw, km_m, tend, u_m, v_m, vsws_m, & w_m ) ELSE CALL advec_v_up( i, j ) CALL diffusion_v( i, j, ddzu, ddzw, km, tend, u, v, vsws, w ) ENDIF CALL coriolis( i, j, 2 ) CALL user_actions( i, j, 'v-tendency' ) …… For the call of tendency-terms on the ibm- regatta, a FORTRAN95-functionality called function-overloading is used to allow different argument lists for the same SUBROUTINE, e.g.: CALL user_actions( before_timestep ) but CALL user_actions(i, j, u-tendency)

10. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 function overloading subroutines have to be embedded in a modular programming structure: MODULE user_actions_mod !-------------------------------------------------- ! Description: ! ------------ ! Execution of user-defined actions before or after ! single timesteps !--------------------------------------------------- PRIVATE PUBLIC user_actions INTERFACE user_actions MODULE PROCEDURE user_actions MODULE PROCEDURE user_actions_ij END INTERFACE user_actions CONTAINS …… !-------------------------- ! Call for all grid points !-------------------------- SUBROUTINE user_actions( location ) …. END SUBROUTINE user_actions !------------------------- ! Call for grid point i,j !-------------------------- SUBROUTINE user_actions_ij( i, j, location ) … END SUBROUTINE user_actions_ij END MODULE user_actions_mod

11. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 Final steps PALM time_integration write_3d_binary write_var_list header user_last_actions cpu_statistics

12. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 writing data for restart run user_last_actions: !– write fields for restart run IF ( write_binary(1:4) == 'true' ) THEN WRITE (14) 'example_array ' ; WRITE (14) example_array WRITE (14) 'example_integer '; WRITE (14) example_integer !-- Set an end mark WRITE ( 14 ) '** end_user_entry ** ' ENDIF use unit 14 to write data

13. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 reading data for restart run user_init: IF … ELSEIF ( TRIM( initializing_actions ) == 'read_fields_from_prior_run' ) THEN ! !-- Read data frtom prior run. READ ( 13 ) field_chr DO WHILE ( TRIM( field_chr ) /= '** end_user_entry **' ) SELECT CASE ( TRIM( field_chr ) ) CASE ('example_array') READ (13) example_array CASE ('example_integer') READ (13) example_integer CASE DEFAULT PRINT*, '+++ user_init: unkown field "', & TRIM( field_chr ), '" find in' PRINT*, ' the data of prior run on PE ', myid STOP END SELECT ! !-- Read next character line READ ( 13 ) field_chr ENDDO ENDIF !initializing_actions

14. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 Hints for new users If not specified, your added commands will be executed by all PEs print*, 'new user' IF ( myid = 0 ) print*, 'old user' END IF new user new user new user new user new user old user Code: Output: (10PEs)

15. Universität Hannover Institut für Meteorologie und Klimatologie Zingst PALM-Seminar July 2004 Entries in user_actions(i,j, *_tendency) are in a twofold loop and will be computed (nxr-nxl)*(nyn-nys) times on each PE. Use the MODULE user for the declaration of variables and arrays, etc. Use SUBROUTINES user_init and user_last_actions to restore and save your data for restart_runs. Hints for new users