1. EGEE-III INFSO-RI-222667 Enabling Grids for E-sciencE www.eu-egee.org EGEE and gLite are registered trademarks António Pina, Ricardo Marques, Bruno Oliveira {pina,boliveira,rjm}@di.uminho.pt Departamento de Informática Universidade do Minho Portugal : FireStation: From Sequential to EGEE-Grid

2. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Civil Protection (CP) applications –programs which model hazard situations (forest fires, floods, etc.) Used by decision makers to get information about how will an emergency evolve. Make the best use of computing resources and data availability. –to allow near real time response 2 Civil Protection

3. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Portuguese NGI FireStation –An integrated system aimed at the simulation of fire growth over a complex topography –demands high computing power and large data set Near-real time visualization Decision Maker support Portal 3 Cross-Fire

4. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 4 FireStation

5. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Developed under the environment of the CAD application MicroStation Integrates 3 modules calculation: –Wind field (Canyon, Nuatmos) –Fire Weather Index of the Canadian System –Fire propagation 5 FireStation

6. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Provides a clear and correct interpretation of the data 3D :Top view, Isometric view, Perspective view 6 6 Graphical Interface

7. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Terrain topology: 7 FS - Terrain

8. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Fuel distribution over the terrain: 8 FS - Fuel

9. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Wind field: 9 FS - Wind

10. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Most important input for fire spread calculation – Influenced by topography and vegetation – a fluid dynamics problem – self contained Fortran code – run as an external program 10 Wind Field Module

11. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Canadian Fire Weather Index (FWI) – A fire danger rating system – How weather/fuel affects fire potential The module allows – broad assessment of large-scale fire potential – estimate moisture content of dead/live fuels 11 Fire Index Module

12. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 12 Wind speed & direction Slope Fuel models Fuel moisture Orientation Fuel load, height, chemical and physical properties Temperature Altitude Relative humidity Meteorology Topography Fuels FIRE BEHAVIOUR MODEL Rate of spread Spotting distance Fire line intensity Probability of ignition Flame height/width Height of dead crown Area, perimeter... Prescription… Overview

13. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Terrain is divided into cells – Properties are assumed to be constant in each cell 13 Topography Fuel models Windfield Slope Orientation Cells Properties

14. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 List of burned cells, and their respective ignition time 14 post-processed to get the fire spread rate and direction, total area burned, etc. FS - Output

15. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Fire growth simulation – contagion between burning and non-burning cells: – computes the time that the fire takes to propagate from the current burning cell to its neighbours – The neighbour cells are assigned a time stamp which represent the instant they will start burning – The non-burning cell with the shortest time assigned becomes the current burning cell 15 FS – a Closer View

16. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 16 while (there are cells marked to burn & not(end simulation) ) { 1.Get the current burning cell 2.Increment the burned cells counter 3.Refresh the total burned area 4.Compute predictions for the neighbour cells to burn, due to burning of the current cell 5.Decide whether or not to continue the simulation } 16 Sequential Algorithm

17. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 17 Fire Growth (I)

18. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 18 Fire Growth (II)

19. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 19 Fire Growth (III)

20. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 20 Fire Growth (IV)

21. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Remove MicroStation  dependency Convert sequential Fire Propagation algorithm into parallel –In a cluster closed environment 21 P-FireStation Goals

22. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Make use of multiple nodes to simulate the fire growth in parallel, assigning each node a subset of cells obtained by partitioning the whole map. Use the Message Passing Interface (MPI), and MPI-IO to inject parallel primitives in the sequential application, in order to create the parallel prototype 22 Main Idea

23. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Simple approach – assign a sub-area to each processor 23 Domain Decomposition

24. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Domain Decomposition 24 Assignment of an area to each processor P0P2P1P3P4 P6P5P7 P8

25. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Rapidly produce a parallel prototype – fire propagation module – no visual interface – keep the core of the sequential code – for evaluation by partners  Integration with GIS  Near real-time visualization 25 Framework - Development

26. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Main phases 1.Analysis of the sequential algorithm 2.Solve the domain decomposition problem 3.Use MPI-IO existent to implement the data stage in/out 4.Create and validate a full functional fire spread parallel algorithm  Using Non-intrusive MPI primitives 26 Phases

27. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Some important functions in the framework: – goToSleep_MPI(): puts a process in sleeping mode until receive a message from other process – receiveMessage_MPI(): Try to receives messages from other processes – sendMsg_MPI(): sends work/allReduce/termination messages to other processes Two derived datatypes: – Coords: represents the position of a cell – Prediction: represents a burning prediction for a cell 27 Parallel Framework

28. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Input/Output stages – MPI-IO predefined functions – local array with ghost boundary area Each process has its own view of the input files – corresponding to a sub-division – compute fire spread in the assigned area Single collective read/write primitives – load/store input data – MPI_Read_all(File, File_View, ….) Comunicate the boundary area with neighbours – MPI Cartesian Topology 28 Parallel I/O

29. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 29 Parallel Fire Growth (I)

30. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 30 Parallel Fire Growth (II)

31. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 31 Parallel Fire Growth (III)

32. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Put P-FireStation on Grid. Using Grid tools and services –worker nodes for the fire growth propagation; –manage large data input/output files; –maintain a metada database of inputs/results to :  avoid repeated simulations;  evaluate simulations under different conditions –support the monitoring of the fire growth  Near-realtime 32 G – FS Goals

33. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 gLite –job description –job submission –output retrieval –data management 33 gLite

34. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 AMGA Metadata Catalogue –Creation/management of an meta data-base –keep track of the results and inputs WatchDog –non-intrusive monitoring tool –publish partial results of simulation in soft real time –allow an soft real time view of the simulation; 34 Tools

35. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 1.Simple – Inputs are passed directly by the user – Outputs are fetched by the user 2.Intermediate – The Job fetches the large inputs from the SE – The Job uploads the results to the SE 3.Final – Job registers the execution in the AMGA database – WatchDog monitors the evolution of the simulation 35 Incremental Development

36. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 36 JobType = “Normal"; CpuNumber = 12; Executable = "fireStation-start-wrapper.sh"; Arguments = "fireStation MPICH"; StdOutput = "std.out"; StdError = "std.err"; InputSandbox = { "$HOME/fireStation/scripts/fireStation-start-wrapper.sh", "$HOME/fireStation/src/fireStation", "$HOME/fireStation/input/ignition.dat", "$HOME/fireStation/input/Control.dat", "$HOME/fireStation/input/terrain.asc.bin", "$HOME/fireStation/input/fuelmodels.fls", "$HOME/fireStation/input/fueldistr.asc.bin", "$HOME/fireStation/input/fire.out.bin", "$HOME/fireStation/scripts/fireStation-hooks.sh", }; OutputSandbox = { "std.err", "std.out", "fireTest.out" }; Simple: JDL File

37. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 37 #!/bin/sh # This function will be called before the MPI executable is started. # You can, for example, compile the executable itself. pre_run_hook () { # Give the application permitions to execute chmod 777 fireStation echo "Finished the pre run." return 0 } # This function will be called before the MPI executable is finished. # A typical case for this is to upload the results to a storage element. # post_run_hook () { return } Simple: Hooks File

38. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 38 JobType = “Normal"; CpuNumber = 12; Executable = "fireStation-start-wrapper.sh"; Arguments = "fireStation MPICH"; StdOutput = "std.out"; StdError = "std.err"; InputSandbox = { "$HOME/fireStation/scripts/fireStation-start-wrapper.sh", "$HOME/fireStation/src/fireStation", "$HOME/fireStation/input/ignition.dat", "$HOME/fireStation/input/Control.dat", "$HOME/fireStation/scripts/fireStation-hooks.sh", "$HOME/fireStation/scripts/fireStation-download-input.sh", "$HOME/fireStation/scripts/fireStation-upload-output.sh" }; OutputSandbox = { "std.err", "std.out", }; Intermediate: JDL File

39. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 39 #!/bin/sh # This function will be called before the MPI executable is started. # You can, for example, compile the executable itself. pre_run_hook () { # Get the Input Files from the Storage Element chmod 777 fireStation-download-input.sh. fireStation-download-input.sh # Give the application permitions to execute chmod 777 fireStation return 0 } Intermediate: pre_run function

40. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 40 #!\bin\bash # Get the Input Files from the Storage Element lcg-cp -v --vo cyclops lfn:/grid/cyclops/fireStation/input/terrain.asc.bin file:./terrain.asc.bin lcg-cp -v --vo cyclops lfn:/grid/cyclops/fireStation/input/fuelmodels.fls file:./fuelmodels.fls lcg-cp -v --vo cyclops lfn:/grid/cyclops/fireStation/input/interpol.out.bin file:./interpol.out.bin lcg-cp -v --vo cyclops lfn:/grid/cyclops/fireStation/input/fueldistr.asc.bin file:./fueldistr.asc.bin Intermediate: get input

41. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 41 # This function will be called before the MPI executable is finished. # A typical case for this is to upload the results to a storage element. post_run_hook () { # Uploading the output form the CE to the SE chmod 77 fireStation-upload-output.sh. fireStation-upload-output.sh return 0 } Intermediate: post_run function

42. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 42 #!/bin/bash # Set of the JobId reference JOBID=‘echo $EDG_WL_JOBID| cut -d "/" -f 4‘ # Uploading the Output File from the WNs to the SE lcg-cr --vo cyclops -l lfn:/grid/cyclops/fireStation/results/fireTest$JOBID.out -d se.cp.di.uminho.pt file://$PWD/fireTest.out Intermediate: upload results

43. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 1.Download large data files from a Storage Element 2.Starts WatchDog job monitoring tool 3.Execute the P-FS, using WatchDog to report fire propagation 4.Register the results in a Grid Storage Elements after termination 5.Add an entry into AMGA metadata base server 43 G-FireStation - Final

44. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 44 JobType = “Normal"; CpuNumber = 12; Executable = "fireStation-start-wrapper.sh"; Arguments = "fireStation MPICH"; StdOutput = "std.out"; StdError = "std.err"; InputSandbox = { (…) "$HOME/fireStation/dummy/watchdog.conf", "$HOME/fireStation/dummy/watchdog.ctrl", "$HOME/fireStation/dummy/watchdog.sh" }; OutputSandbox = { "std.err", "std.out", }; Final: JDL File

45. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 45 # This function will be called before the MPI executable is started. # You can, for example, compile the executable itself. pre_run_hook () { # Get the Input Files from the Storage Element chmod 777 fireStation-download-input.sh. fireStation-download-input.sh # prepare and start the watchdog chmod +x watchdog.sh chmod +x watchdog.ctrl chmod +x watchdog.conf./watchdog.ctrl start # Give the application permitions to execute chmod 777 fireStation return 0 } Final: pre_run function

46. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 46 post_run_hook () { # Uploading the output form the CE to the SE chmod 77 fireStation-upload-output.sh. fireStation-upload-output.sh #stop and wait the watchdog complete./watchdog.ctrl stop # Synchronising Amga database with the SE chmod 777 synchronize-amga-se.sh. synchronize-amga-se.sh jobid.amga return 0 } Final: post_run function

47. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 47 file=$1 #1) Set environment variables for SE SE_HOST=“se.cp.di.uminho.pt" SEDir="/grid/cyclops/fireStation/results“ AMGA_DIR="/test/fireStation“ #3) Set the JobId reference # Get the lfn reference from file \"jobid.amga\ echo LFN=‘cat $file‘ echo " Value of lfn: $LFN“ #4) Synchronising AMGA with SE AREABURNED=42 WINDKIND="South-WestWind" mdcli addentry $AMGA_DIR/$LFN date \"$DATE\" areaName \"$AREANAME\" areaSize $AREASIZE areaBurned $AREABURNED windKind \"$WINDKIND\” Final: AMGA script

48. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 48 Both the Parallel and the Grid version were implemented and executed successfully It was showed that the Grid can give a good response to improve a parallel application with : –large input/output demands –soft real time monitoring –support to large metadata database of simulations Conclusions

49. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Paralelization/Gridification  Wind field models  Using MPI/MP-IO OGC-WS integration  WCS/WMS/WFS  WPS Graphical user interface gvSIG- GIS full feature desktop Decision-support portal Genius under evaluation 49 Ongoing Work

50. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Future: Portal 50

51. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Future: Workflow 51

52. Enabling Grids for E-sciencE EGEE-III INFSO-RI-222667 Thank you 52