1. Kento Aida, Tokyo Institute of Technology Ninf-G: GridRPC System Tokyo Institute of Technology Kento Aida

2. Kento Aida, Tokyo Institute of Technology GridRPC System A simple RPC-based programming model for the Grid Client invokes remote libraries installed on remote servers on the Grid. Key property: EASE OF USE Very simple RPC API Existing libraries and apps on remote servers server library server library data result data result client

3. Kento Aida, Tokyo Institute of Technology GridRPC System (cont’d) Programming model middleware between “the Grid toolkit” and Application Bases for more complex form of Grid computing Grid toolkit (e.g. Globus) GridRPC System Application

4. Kento Aida, Tokyo Institute of Technology GridRPC – What’s going on? - Activities at the GGF Standard GridRPC API Provide several reference implementations System developments GridSolve, Ninf-G/G2 Applications Monte Carlo Simulation – MCell (Netsolve) REXMC (Ninf), Weather Forecasting (Ninf-G) BMI, SDPA Apps (Ninf/Ninf-G) SCLAPACK for Netsolve/Ninf/Ninf-G

5. Kento Aida, Tokyo Institute of Technology Ninf Project 1994199720002003 Ninf project launched Release Ninf version 1 Start collaboration with NetSolve team Ninf-G development Release Ninf-G version 0.9 Release Ninf-G version 1.0 Standard GridRPC API proposed 1 st GridRPC WG at GGF7 Implement formal spec. of GridRPC Collaborator: AIST, University of Tsukuba, Tokyo Institute of Technology, Tokyo Electronic University, Ochanomizu University, Kyoto University

6. Kento Aida, Tokyo Institute of Technology What is Ninf-G? A software package which supports programming and execution of Grid applications using GridRPC. Ninf-G includes C/C++, Java APIs, libraries for software development IDL compiler for stub generation Shell scripts to compile client program build and publish remote libraries sample programs manual documents

7. Kento Aida, Tokyo Institute of Technology Ninf-G: Features At-a-Glance Ease-of-use, client-server, Numerical-oriented RPC system No stub information at the client side User’s view: ordinary software library Built on top of the Globus Toolkit Uses GSI, GRAM, MDS, GASS, and Globus-IO Supports various platforms Ninf-G is available on Globus-enabled platforms Client APIs: C/C++, Java

8. Kento Aida, Tokyo Institute of Technology Architecture of Ninf-G Client GRAM Invoke Executable Connect back IDL file Numerical Library IDL Compiler Remote Library Executable Generate Interface Request Interface Reply Server sideClient side fork GRIS Interface Information LDIF File retrieve Globus-IO

9. Kento Aida, Tokyo Institute of Technology Building Ninf-G remote libraries 1.Provide library programs and interface info. written by Interface Description Language (Ninf-G IDL) 2.Compile the Ninf-G IDL with Ninf-G IDL compiler (ns_gen) ns_gen generates stub source files and a makefile (.mak). 3.Compile the stub source files by the makefile It generates Ninf-G executables and LDIF files. LDIF files are used to register Ninf-G remote libs information to GRIS. 4.Publish the Ninf-G remote libraries The LDIF files are copied to the information service.

10. Kento Aida, Tokyo Institute of Technology Building Ninf-G remote libraries (cont’d) GRIS.mak Ninf-G IDL file.idl ns_gen _stub_goo.c _stub_goo _stub_bar.c _stub_bar _stub_foo.c _stub_foo Library program libfoo.a Library program libfoo.a ::goo.ldif ::bar.ldif ::foo.ldif GRAM make – f.mak

11. Kento Aida, Tokyo Institute of Technology Sample Ninf-G IDL Matrix Multiply Module matrix; Define dmmul (IN int n, IN double A[n][n], IN double B[n][n], OUT double C[n][n]) “Matrix multiply: C = A x B“ Required “libmmul.o” Calls “C” dmmul(n, A, B, C); Module matrix; Define dmmul (IN int n, IN double A[n][n], IN double B[n][n], OUT double C[n][n]) “Matrix multiply: C = A x B“ Required “libmmul.o” Calls “C” dmmul(n, A, B, C);

12. Kento Aida, Tokyo Institute of Technology Building and Running Client Programs 1.Write client programs using Ninf-G API 2.Write a configuration file in which runtime environments can be described 3.Compile and link with the Ninf-G client compile driver (ns_client_gen) 4.Run grid-proxy-init command 5.Run the program

13. Kento Aida, Tokyo Institute of Technology Client Program : grpc_initialize()/* initialization */ : grpc_function_handle_init()/* creation of function handles */ : grpc_call()/* call to remote library */ : grpc_wait()/* synchronization */ : grpc_finalize/* finalization */ :

14. Kento Aida, Tokyo Institute of Technology GridRPC API - Initialize/Finalize - Any invocation to remote libraries should be performed between the initialization and the finalization. int grpc_initialize( char * config_file_name); reads the configuration file and initialize client. int grpc_finalize(); Frees resources (memory, etc.)

15. Kento Aida, Tokyo Institute of Technology GridRPC API – Function Handles - Function Handles should be created before calling remote libraries. int grpc_function_handle_default ( grpc_function_handle_t * handle, char * func_name); Initializes a function handle for the function using default host and port. int grpc_function_handle_init ( grpc_function_handle_t * handle, char * host_name, int port, char * func_name); Initializes a function handle for the function along with the specified host and port. int grpc_function_handle_destruct ( grpc_function_handle_t * handle); Destructs the function handle

16. Kento Aida, Tokyo Institute of Technology GridRPC API – Blocking RPC - int grpc_call ( grpc_function_handle_t *,...); Invokes a blocking RPC using the specified function handle. The rest of the arguments should be the same sequence with the Ninf-G remote library. grpc_c all Client Server

17. Kento Aida, Tokyo Institute of Technology GridRPC API - Nonblocking RPC - int grpc_call_async (grpc_function_handle_t *,...); Invokes a non-blocking RPC using the specified function handle. The rest of the arguments should be the same sequence with the Ninf-G remote library. grpc_call_a sync Client ServerA ServerB grpc_call_a sync grpc_wai t_all

18. Kento Aida, Tokyo Institute of Technology GridRPC API - Wait Functions - int grpc_wait (int sessionID); Waits outstanding RPC specified by sessionID int grpc_wait_and (int * idArray, int length); Waits all outstanding RPCs specified by an array of sessionIDs int grpc_wait_or ( int * idArray, int length, int * idPtr); Waits any one of RPCs specified by an array of sessionIDs. int grpc_wait_all ( ); Waits until all outstanding RPCs are completed. int grpc_wait_any ( int * idPtr); Waits any one of outstanding RPCs.

19. Kento Aida, Tokyo Institute of Technology GridRPC API - Session Control - int grpc_probe (int sessionID); probes the job specified by SessionID whether the job has been completed. Returns 1 if the job is done and 0 is returned if the job is still running. int grpc_cancel (int sessionID); Cancel outstanding RPC specified by sessionID. Returns GRPC_OK (success) or GRPC_ERROR (failure)

20. Kento Aida, Tokyo Institute of Technology How the server can be specified? Server is determined when the function handle is created. grpc_function_handle_init(); hostname is given as the second argument grpc_function_handle_default(); hostname is specified in the client configuration file which must be passed as the first argument of the client program. Notes: Ninf-G Ver.1 is designed to focus on GridRPC core mechanism and it does not support any brokering/scheduling functions.

21. Kento Aida, Tokyo Institute of Technology Sample Configuration File # call remote library on UME cluster serverhost = ume.hpcc.jp # grd jobmanager is used to launch jobs jobmanager = jobmanager-grd # query to ApGrid GIIS ldaphost = mds.apgrid.org ldapport = 2135 vo_name = ApGrid # get detailed log loglevel = 3 # call remote library on UME cluster serverhost = ume.hpcc.jp # grd jobmanager is used to launch jobs jobmanager = jobmanager-grd # query to ApGrid GIIS ldaphost = mds.apgrid.org ldapport = 2135 vo_name = ApGrid # get detailed log loglevel = 3

22. Kento Aida, Tokyo Institute of Technology Example Compute PI using Monte-Carlo Method Generate a large number of random points within the square region that exactly encloses a unit circle (1/4 of a circle) Count the number of points generated within a circle PI = 4 p

23. Kento Aida, Tokyo Institute of Technology Compute PI - Server Side - Module pi; Define pi_trial ( IN int seed, IN long times, OUT long * count) "monte carlo pi computation" Required "pi_trial.o" { long counter; counter = pi_trial(seed, times); *count = counter; } long pi_trial (int seed, long times) { long l, counter = 0; srandom(seed); for (l = 0; l < times; l++) { double x = (double)random() / RAND_MAX; double y = (double)random() / RAND_MAX; if (x * x + y * y < 1.0) counter++; } return counter; } pi.idlpi_trial.c

24. Kento Aida, Tokyo Institute of Technology Compute PI - Client Side- #include "grpc.h" #define NUM_HOSTS 8 char * hosts[] = {"host00", "host01", "host02", "host03", "host04", "host05", "host06", "host07"}; grpc_function_handle_t handles[NUM_HOSTS]; main(int argc, char ** argv){ double pi; long times, count[NUM_HOSTS], sum; char * config_file; int i; if (argc < 3){ fprintf(stderr, "USAGE: %s CONFIG_FILE TIMES \n", argv[0]); exit(2); } config_file = argv[1]; times = atol(argv[2]) / NUM_HOSTS; /* Initialize */ if (grpc_initialize(config_file) != GRPC_OK){ grpc_perror("grpc_initialize"); exit(2); } /* Initialize Function Handles */ for (i = 0; i < NUM_HOSTS; i++) grpc_function_handle_init(&handles[i], hosts[i], port, "pi/pi_trial"); for (i = 0; i < NUM_HOSTS; i++) /* Asynchronous RPC */ if (gprc_call_async(&handles[i], i, times, &count[i]) == GRPC_ERROR){ grpc_perror("pi_trial"); exit(2); } /* Wait all outstanding RPCs */ if (grpc_wait_all() == GRPC_ERROR){ grpc_perror("wait_all"); exit(2); } /* Display result */ for (i = 0, sum = 0; i < NUM_HOSTS; i++) sum += count[i]; pi = 4.0 * ( sum / ((double) times * NUM_HOSTS)); printf("PI = %f\n", pi); /* Finalize */ grpc_finalize(); }

25. Kento Aida, Tokyo Institute of Technology Compute PI - Client Side- (cont’d) Creation of function handles #define NUM_HOSTS 8 char * hosts[] = {"host00", "host01", "host02", "host03", "host04", "host05", "host06", "host07"}; grpc_function_handle_t handles[NUM_HOSTS]; /* Initialize Function Handles */ for (i = 0; i < NUM_HOSTS; i++) grpc_function_handle_init(&handles[i], hosts[i], port, "pi/pi_trial");

26. Kento Aida, Tokyo Institute of Technology Compute PI - Client Side- (cont’d) RPC and Wait for (i = 0; i < NUM_HOSTS; i++) /* Asynchronous RPC */ if (gprc_call_async(&handles[i], i, times, &count[i]) == GRPC_ERROR){ grpc_perror("pi_trial"); exit(2); } /* Wait all outstanding RPCs */ if (grpc_wait_all() == GRPC_ERROR){ grpc_perror("wait_all"); exit(2); }

27. Kento Aida, Tokyo Institute of Technology More Info. Ninf home page http://ninf.apgrid.org GGF http://www.ggf.org/ Contacts ninf@apgrid.org Acknowledgements: Yoshio Tanaka, Hidemoto Nakada, other Ninf project members