1. GridSuperscalar A programming model for GRID applications José Mª Cela cela@ac.upc.es cela@ciri.upc.es

2. What is GRID? Conected heterogeneous resources AC Campus Castelldefels LCFIB Campus Nord CEPBA Campus Nord What we have? 500GFlops 40PB disk space 10TB RAM... Transparent access to basic resources What we want?

3. Basic brick in a GRID SERVICESLIBRARIES JAVA C (Unix) TCP/IP OPEN CODE DOCUMENTATION UTILTIES Globus Toolkit 2.x

4. What is GLOBUS? Globus Resource Allocation Manager (GRAM) Grid Security Infrastructure (GSI) Monitoring and Discovery Service (MDS) Global Access to Secondary Storage (GASS) Manage and execute processes Secure Access Access to sytem information Data transfer CLIENT SERVER LIBRARIES UTILITIES

5. Security Autentification Resource GATEKEEPER GRID User -> Local User GRID-MAPFILE Autorization Public Key Private Key Resource certificate Public Key Private Key Temporal Proxy GRID User Certificate

6. Monitoring Systems GISS-ALL... GRIS (Grid Resource Information Service) GISS (Grid Index Information Service)

7. Data transfer RECURSO GSIFTP GRID User -> Local User GRID-MAPFILE RESOURCE CERTIFICATION Temporal Proxy GSIFTP - gsiftp:// GASS - https:// GASS OK! Local User Temporal Proxy Always available Slow speed High Speed It has implementation problems

8. Execute processes RSL SCRIPT '&(parameter=valorY) (parameter1=valorX)...' executable= arguments= stdin= stdout= queue= environment= directory=... GRAM CLIENT Resource WITHOUT it GRAM SERVER No Grid Job Grid Job LoadLeveler, NQE GRAM SERVER Resource WITH Load Balancing Control

9. Execute processes main() { globus_module_activate( GLOBUS_GRAM_CLIENT_MODULE ); err = globus_gram_client_callback_allow( Callback, NULL, &contact ); if (err != GLOBUS_SUCCESS) ERROR(); desc = "(&executable=/home/ac/cela/Dimemas)(arguments= -o kk.out file.cfg) (scratch_dir=/scratch/ac/cela)(directory=/home/ac/cela/) (file_stage_in=(https://kandake:20352/home/ac/cela/file.cfg $(SCRATCH_DIRECOTRY)/file.cfg) (https://kandake:20352/home/ac/cela/matrix.trf $(SCRATCH_DIRECOTRY)/matrix.trf)) (file_stage_out=($(SCRATCH_DIRECOTRY)/kk.out https://kandake:20352/home/ac/cela/kk.out) (environment=(DIMEMAS_HOME/usr/local/cepba-tools)(TMPDIR $(SCRATCH_DIR)))"; err = globus_gram_client_job_request( contact, desc, GLOBUS_GRAM_PROTOCOL_JOB_STATE_DONE, callback_contact, &job_contact ); if (err != GLOBUS_SUCCESS) ERROR(); err = globus_gram_client_job_status( job_contact, &job_status, &res ); if (err != GLOBUS_SUCCESS) ERROR(); while (1) { globus_poll(); globus_poll_blocking(); } globus_gram_client_job_contact_free( job_contact ); globus_module_deactivate( GLOBUS_GRAM_CLIENT_MODULE ); }

10. GridSuperscalar A programming paradigm which allows: –Reduce the complexity of developing Grid Applications to the minimum –Automatic task parallelization over a GRID environment Basic idea: superscalar processors –Sequential control flow –Well defined object name space, I/0 arguments to operation –Automatic construction of precedence DAG –Renaming –Forwarding –DAG scheduling, locality management –Prediction, Speculation

11. GridSuperscalar basis Code: –Sequential application in C/C++/Fortran90 with calls the GridSuperscalar run-time (Execute call) Run-time performs: –Task identification (based on Execute primitive) –Data dependency analysis: files are the objects –Data dependence graph creation –Task scheduling based on the graph –File renaming to increase graph concurrency –File forwarding

12. GridSuperscalar behavior overview for (i=0; i<N; i++ { Execute(T1, “file1.txt”, “file2.txt”); Execute(T2, “file4.txt”, “file5.txt”); Execute(T3, “file2.txt”, “file5.txt”, “file6.txt”); Execute(T4, “file7.txt”, “file8.txt”); Execute(T5, “file6.txt”, “file8.txt”, “file9.txt”); } Application code T1 0 T2 0 T3 0 T4 0 T5 0 T1 1 T2 1 T3 1 T4 1 T5 1 T1 2 … Grid

13. GridSuperscalar: user interface Actions to do when developing an application 1.Task definition: identify those subroutines/programs to be executed in the Grid 2.Tasks’ interface definition: input/output files and input/output generic scalars 3.Write the sequential program using calls to the GridSuperscalar primitives (Execute) Instruction set definition

14. GridSuperscalar: code example Simple optimization search example: –perform Neval simulations –recalculate range of parameters –end when goal is reached 1.Tasks: FILTER, DIMEMAS, EXTRACT 2.Parameters (current syntax): FILTER 1 2 1 0 DIMEMAS 2 0 1 0 EXTRACT 1 0 1 0 3.Sequential code

15. GridSuperscalar: code example Range = initial_range(); while (!goal_reached() && (j<MAX_ITERS)) { for (i= 0; i< Neval; i++) { L[i] = gen_rand_L_within_current_range(range); BW[i] = gen_rand_BW_within_current_range(range); Execute( FILTER, “bh.cfg”, L[i], BW[i], “bh_tmp.cfg”); Execute( DIMEM, “bh_tmp.cfg”,“trace.trf”, “dim_out.txt”); Execute( EXTRACT, “dim_out.txt”, “final_result.txt”, “final_result.txt”); } GS_Barrier(); generate_new_range(“final_result.txt”, &range); j++; }

16. GridSuperscalar: code example Worker: switch(atoi(argv[2])) { case FILTER: res = filter(argc, argv); break; case DIMEM: res = dimemas_funct(argc, argv); break; case EXTRACT:res = extract(argc, argv); break; default: printf("Wrong operation code\n"); break; }

17. GridSuperscalar run-time: task graph generation Range = initial_range(); while (!goal_reached() && (j<MAX_ITERS)){ for (i=0; i< Neval; i++){ L[i] = gen_rand_L_within_current_range(range); BW[i] = gen_rand_BW_within_current_range(range); Execute( FILTER, “bh.cfg”, L[i], BW[i], “bh_tmp.cfg” ); Execute( DIMEMAS, “bh_tmp.cfg”,“trace.trf”, “dim_out.txt” ); Execute( EXTRACT, “dim_out.txt”, “final_result.txt” ); } GS_Barrier(); generate_new_range(“final_result.txt”, &range); j++; } FILTER DIMEMAS EXTRACT FILTER DIMEMAS EXTRACT FILTER DIMEMAS EXTRACT … Neval BARRIER …

18. GridSuperscalar: task scheduling FILTER DIMEMAS EXTRACT FILTER DIMEMAS EXTRACT FILTER DIMEMAS EXTRACT … BARRIER … CIRI Grid

19. GridSuperscalar: task scheduling FILTER DIMEMAS EXTRACT FILTER DIMEMAS EXTRACT FILTER DIMEMAS EXTRACT … BARRIER … CIRI Grid Additional function: –GS_barrier

20. “f1_2” “f1_1” GridSuperscalar: renaming Additional functions: –GS_open, GS_close T1_1 T1_2 T1_3 T2_1 T2_2 T2_3 TN_1 TN_2 TN_3 … “f1”

21. GridSuperscalar: file forwarding T1 T2 f1 T1 T2 f1 (by socket) Prototype implemented with Dyninst Allows to execute different tasks on different hardware resources Initial tests High overhead due to worker behavior mutation Future tests with new Dyninst version 4.0

22. Grid superscalar: current Globus implementation Previous prototype over Condor and MW Current prototype over Globus 2.x, using the API File transfer, security, … provided by Globus Task submission –globus_gram_client_job_request Asynchronous end of task synchronization –Asynchronous state-change callbacks mechanism provided by Globus –globus_gram_client_callback_allow –callback_func function, provided by the programmer

23. NAS GRID benchmarks –Designed to provide an objective measure of GRID systems –At the present it is only a paper and pencil description –The basic executables are the NAS Parallel Benchmarks BT, SP, LU => Solvers MG => Post-processor FT => Data visualization MF => Interpolation filter –Only the data flow graphs are defined 4 data flow graphs proposed (ED, HC, VP, MB) GRID Performance Measurement

24. Embarrassingly Distributed (ED) –Parameter studies GRID Performance Measurement Launch Report SP

25. Helical Chain (HC) –A set of flow calculations GRID Performance Measurement Launch Report MF BTSPLU MF BTSPLU MF BTSPLU MF

26. Visualization Pipe (VP) –Solver + Post-processor + Visualization module GRID Performance Measurement Launch Report MF BTMGFT MF BTMGFT MF BTMGFT MF

27. Mixed Bag (MB) –VP with asymmetry GRID Performance Measurement Launch Report LU MG FT MF

28. Original script eval "(./bin/bt.$CLASS <<-EOF ) | $THROWIT /dev/null $ASCII $NAME $CLASS $ITER $WIDTH $DEPTH $PID $VERBOSE EOF" GRID Performance Measurement

29. Present code in our programing model sprintf( genIn, "%s %s %d %d %d %d %d %d", NAME, CLASE, ascii, iter, width, depth, pid, verbose); Execute( BT, "HC_BT_IN", genIn, "HC_BT_OUT" ); GRID Performance Measurement