1 SECG/CAS/eScience-2007 Regional Weather Modeling and Grid Computing at C-DAC Computational Atmospheric Sciences Team, Centre for Development of Advanced Computing, Pune

2 SECG/CAS/eScience-2007 Scope Background Mausam Grid Case Study –Regional Coupled Climate Model on Grid –Grid-based Meteorology-Air Quality Modeling

3 SECG/CAS/eScience-2007 Computational Atmospheric Sciences Activities Computational Research Workflow Environment Development Technology Development Parallel Programming Model Porting, Optimisation & Simulations Grid Computing  Joint Collaborative Research  Turnkey solutions  Contract Projects  Consultancy

4 SECG/CAS/eScience-2007 C-DAC’s Vision – Mausam Grid

5 SECG/CAS/eScience-2007 Mausam Grid A portal for Atmospheric & Environmental Sciences Community Multimodel, Multidata, Query based Problem Solving Environment Secured Access Based on Garuda Infrastructure Regional Models – RegCM, WRF, RSM Ocean Models – ROMS, HYCOM Air Quality Models – AERMOD, WRF-CHEM

6 SECG/CAS/eScience-2007 Resources Specifies Information regarding the available resources on Garuda Status of resources, such as Cluster Name, availability of Processors and nodes are displayed User can take decision regarding no. of processors to specify based on machine availability.

7 SECG/CAS/eScience-2007 Job Submission Specify the model from the drop down list. (Users can run various atmospheric models ported by the team of experts across Garuda Infrastructure.) User has to specify the type of resource required. (e.g Linux/aix) Easy to remember Job ID. (Jobs are recognized by their name) Selection of resources is automatic, based on the availability of resources of specified type. Can provide multiple input & output files Can specify command line & environmental variables.

8 SECG/CAS/eScience-2007 CASE STUDY RegCM3 – ROMS WRF – ROMS WRF - AERMOD Reference: tar.gz

9 SECG/CAS/eScience-2007 Regional Atmosphere-Ocean Coupled Model RegCM3 ROMS  EUIndia Project: Development of regional coupled model on Grid  Regional Climate Model (RegCM3): Atmospheric  Regional Ocean Modeling System (ROMS): Ocean  Development of online coupler : RegCM3 and ROMS  Coupled System tested on EGEE grid  Long term runs and Analysis in progress  Coupled System used for Climate simulations

10 SECG/CAS/eScience-2007 WRF - ROMS Coupling The Models coupled using Model Coupling Toolkit MCT consists of a set of distributed-memory routines based on MPI WRF Domain 161 by 161 lateral points and 30 levels ROMS domain is 200 by 200 horizontal points and 30 vertical levels ROMS supplies the SST to WRF WRF provides wind stresses and heat to ROMS Vortex generation is modeled by initiating a tropical storm like vertical wind field in gradient wind balance. The source code for this can be downloaded from roms/wrf_roms-1.2.tar.gz

11 SECG/CAS/eScience-2007 Meteorological data critical input for Air Quality models High resolution NWP models used for AQ model initialization C-DAC: Development of Coupler for WRF and AERMOD Simulation time/day- WRF:30 min, AERMOD:15 min Simulation time – 6 day (serial flow) : 270 min Meteorology Air Quality Coupled System

12 SECG/CAS/eScience-2007 Heterogeneous Grid Infrastructure Exploit the various Windows & Linux based resources available in the research lab Middleware used: –Globus Toolkit –Alchemi (Window based)

13 SECG/CAS/eScience-2007 Schematic Presentation of Meteorology-Air Quality Grid System WRF - AERMOD Grid - Flow Contact:

14 SECG/CAS/eScience-2007 Seismic Tomographic Inversion on EU-India Grid Seismic Data Processing Team Centre for Development of Advanced Computing Pune University Campus, Pune (India)

15 SECG/CAS/eScience-2007  The word "tomography" derived from the Greek tomos (slice) and graphia (describing). The method used in: Medicine Astronomy Geology Materials Science What is Tomography ?

16 SECG/CAS/eScience-2007 Seismic inversion provides method for direct estimation of velocity distribution in the subsurface of the earth using initial earth model as guess. It generates a cross-sectional picture of an object by utilizing the object’s response to external energy source. Seismic applications like mining, dam foundation building, ground water probing etc. Seismic Tomographic Inversion

17 SECG/CAS/eScience-2007 Cross-hole test method – field setup Ray diagram for different positions of sources and receivers Velocity distribution in different pixels P-wave velocity contour along the vertical plane between source and receiver borehole Seismic Tomographic Survey

18 SECG/CAS/eScience-2007 The inversion scheme is Real-coded Genetic Algorithm. Hybrid Island model (Chipperfield and Fleming, 1996) is adopted for the parallel implementation of the genetic algorithms Seismic Tomographic Inversion

19 SECG/CAS/eScience-2007 EU-INDIA GRID DEMONSTRATION

20 SECG/CAS/eScience-2007 Files required for job submission: gatom4grid_crossplot_N (executable) grid_job.param (job card) init_vel_L.mod (initial model file) Traveltime_L.dat (field data file) TomoOutput.tar (application specific output file) OutputDirectory.sh

21 SECG/CAS/eScience-2007 Tomo.jdl [ Executable = “gatom4grid_crossplot_N “; JobType = “MPICH”; NodeNumber = 4; StdOutput = “std.out”; StdError = “std.out”; InputSandbox = {“gatom4grid_crossplot_N “, “grid_job.param”, “vel_L.mod”,”Traveltime.dat”,”init_vel_L.mod”,”OutputDirectory.sh”}; OutputSandbox = {“std.out”,”std.err”,”TomoOutput.tar”}; ]

22 SECG/CAS/eScience-2007 Proxy Certification voms-proxy-init -voms euindia Cannot find file or dir: /home/jvratnam/.glite/vomses Your identity: /C=TW/O=AP/OU=GRID/CN=Akshara Enter GRID pass phrase: Creating temporary proxy Done Contacting voms2.cnaf.infn.it:15010 [/C=IT/O=INFN/OU=Host/L=CNAF/CN=voms2.cnaf.infn.it] "euindia" Done Creating proxy Done Your proxy is valid until Sat Sep 22 01:52: EUIndia Grid Submission

23 SECG/CAS/eScience-2007 Job Submission glite-wms-job-submit –a -o id Tomo.jdl Selected Virtual Organisation name (from proxy certificate extension): euindia Connecting to host eu-india-02.pd.infn.it, port 7772 Logging to host eu-india-02.pd.infn.it, port 9002 ============ glite-job-submit Success ======================== The job has been successfully submitted to the Network Server. Use glite-job-status command to check job current status. Your job identifier is: - The job identifier has been saved in the following file: /home/jvratnam/SEISMIC/id

24 SECG/CAS/eScience-2007 Job Status glite-wms-job-status -i id ************************************************************* BOOKKEEPING INFORMATION: Status info for the Job : 02.pd.infn.it:9000/QcCFmI6l-IRXXcVT2fIF5A Current Status: Scheduled Status Reason: Job successfully submitted to Globus Destination: ictpgrid-ce-1.ictp.it:2119/jobmanager-pbs-euindia Submitted: Fri Sep 21 13:53: CEST *************************************************************

25 SECG/CAS/eScience-2007 Job Status glite-wms-job-status –i id ************************************************************ BOOKKEEPING INFORMATION: Status info for the Job : IRXXcVT2fIF5A Current Status: Running Status Reason: Job successfully submitted to Globus Destination: ictpgrid-ce-1.ictp.it:2119/jobmanager-pbs-euindia Submitted: Fri Sep 21 13:53: CEST *************************************************************

26 SECG/CAS/eScience-2007 Job Status glite-wms-job-status –i id ************************************************************ BOOKKEEPING INFORMATION: Status info for the Job : IRXXcVT2fIF5A Current Status: Done (Success) Exit code: 0 Status Reason: Job terminated successfully Destination: ictpgrid-ce-1.ictp.it:2119/jobmanager-pbs-euindia Submitted: Fri Sep 21 13:53: CEST *************************************************************

27 SECG/CAS/eScience-2007 Retrieving of the Output glite-wms-job-output --dir output –i id Retrieving files from host: eu-india-02.pd.infn.it ( for india-02.pd.infn.it:9000/e_l5kAbe0rkeg2_JpVEmag ) ************************************************************************ ********* JOB GET OUTPUT OUTCOME Output sandbox files for the job: - have been successfully retrieved and stored in the directory: /output ************************************************************************

28 SECG/CAS/eScience-2007 tar -xvf TomoOutput.tar TomoOutput/./ TomoOutput/./job.status TomoOutput/./output.dump TomoOutput/./error.crv TomoOutput/./fitness.crv TomoOutput/./gen_err.out TomoOutput/./vel_mod_L_iter_1.GRD TomoOutput/./pre_rectime_L_1.dat TomoOutput/./obs_pre_rectime_L_1.dat TomoOutput/./vel_mod_L_iter_2.GRD TomoOutput/./pre_rectime_L_2.dat TomoOutput/./obs_pre_rectime_L_2.dat

29 SECG/CAS/eScience-2007 RESULTS

30 SECG/CAS/eScience-2007 Output Data :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: :::::::: PARALLEL GENETIC ALGORITHM ::::::: :::::::: For Seismic Traveltime Tomography ::::::: :::::::: (2D Elastic Impedance Modelling) ::::::: :::::::: ============================ ::::::: :::::::: (c) Seismic Data Processing Group ::::::: :::::::: C-DAC, Pune ::::::: :::::::: All rights reserved. ::::::: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: GA GA Type : � No. of Runs : 1 No. of Generations : 10 No. of Populations : 10 Probability Crossover : Probability Mutation : Bits per Gene : 0 Tournament Size : 2 Starting seed : Crossover type : b eta and mutation : OpenMod

31 SECG/CAS/eScience-2007 Result of L shape Model                       (c) (a) (b) (d) (a) initial velocity model (b) reconstructed model (c) original model (d) error vs generation graph for a L- shaped model (velocity of 2.5km/sec and 3.5 km/sec) with the background velocity of 3.0km/sec.

32 SECG/CAS/eScience-2007 Thank you