1. Cracow Grid Workshop ’03, 27-29.10.2003 Grid-based System for Flood Forecasting Ladislav Hluchy Institute of Informatics SAS in co-operation with Water Research Institute, Vah River Authority and Slovak Hydrometeorological Institute Slovakia hluchy.ui@savba.sk

2. Cracow Grid Workshop ’03, 27-29.10.2003 Outline Introduction Flood Forecasting Grid infrastructure for Flood Forecasting Use cases Grid-based Implementation Results Conclusion

3. Cracow Grid Workshop ’03, 27-29.10.2003 Flood Forecasting one of the Geospatial Applications Applications that use data from Geographic Information System (GIS) Typical applications: flood forecasting, fire simulations, environmental risk management etc.

4. Cracow Grid Workshop ’03, 27-29.10.2003 Flood Forecasting Topical problem: floods have caused widespread damages in the recent years Common interest: many countries threatened Many potential users: governments, flood crisis teams, insurance companies, public, Requires Grid technology

5. Cracow Grid Workshop ’03, 27-29.10.2003 ANFAS Architecture ANFAS

6. Cracow Grid Workshop ’03, 27-29.10.2003 Integration to ANFAS core Machine 1 ANFAS Core Server currently hosted by EADS-MS&I Web Machine 2 hosting RPS Controller i-cluster LAN Use of the i-cluster in the ANFAS system ANFAS

7. Cracow Grid Workshop ’03, 27-29.10.2003 SMS/FESWMS FESWMS has been developed under funding by the U.S. Federal Highways Administration (FHWA) FESWMS is specifically suited for modeling regions involving flow control structures, such as are encountered at the intersection of roadways and waterways. Specifically, the FESWMS model allows the user to include weirs, culverts, drop inlets, and bridge piers into a standard 2D finite element model. As there is highway planned at the Vah River pilot site in Slovakia, the choice of FESWMS model is important SMS provides graphical tools for defining these structures and controlling analysis using the FESWMS model. Both pre- and post- processing capabilities are included in the interface. ANFAS

8. Cracow Grid Workshop ’03, 27-29.10.2003 Detailed FESWMS structures Input files Finite element Nonlinear solver write solution to the file Solution file Update solution Generating matrix Linear solver OK Nonlinear solverSolution schema ANFAS

9. Cracow Grid Workshop ’03, 27-29.10.2003 Parallel matrix generation Generating partial matrix PARALLEL LINEAR SOLVER Updating solution ANFAS

10. Cracow Grid Workshop ’03, 27-29.10.2003 Implementation issues Real flood modeling software is much more complicated than its mathematical model: –Mathematical model of flood is well-known (partial differential equations  finite elements  nonlinear solver  linear solver) –Real software has to deal with Input processing: different types input data, different variations of each type, different formats of each variation Special cases: wetting/drying, raining/evaporation, special constructions (bridges, dams, culverts), wind effect, … Calibration of results Graphical user interface (GUI), visualisation Error checking, documentation As the result, source code of real software may be hundreds times longer than source code of mathematical model ANFAS

11. Cracow Grid Workshop ’03, 27-29.10.2003 Remote processing Pre-processing Post-processing Processing input data Save solutions Parallel computational kernel Remote processing ANFAS

12. Cracow Grid Workshop ’03, 27-29.10.2003 Planned highway in the Váh pilot site ANFAS

13. Cracow Grid Workshop ’03, 27-29.10.2003 Main part affected by highway LIDAR+highway position Bytca city Predmier village ANFAS

14. Cracow Grid Workshop ’03, 27-29.10.2003 Predmier village in orthophotomap ANFAS

15. Cracow Grid Workshop ’03, 27-29.10.2003 Predmier village in LIDAR ANFAS

16. Cracow Grid Workshop ’03, 27-29.10.2003 TIN network at Predmier ANFAS

17. Cracow Grid Workshop ’03, 27-29.10.2003 Scenario: Water level for current terrain situation (Q-100-year) Water depth ANFAS

18. Cracow Grid Workshop ’03, 27-29.10.2003 Scenario: Water level for highway without bridges (Q-100-year) Water depth Water level is about 70cm higher than for situation without highway ANFAS

19. Cracow Grid Workshop ’03, 27-29.10.2003 Scenario: Water level for highway with 2 bridges (Q-100-year) Water depth Water level is about 30cm higher than for situation without highway ANFAS

20. Cracow Grid Workshop ’03, 27-29.10.2003 Why Grid? Cooperation: requires cooperation between many organizations (meteorological institutes, river authorities) from many countries Data management: needs large amount of data of different sources, different owners, different countries, different access right Computation power: forecasting require large computational power for modeling and simulation

21. Cracow Grid Workshop ’03, 27-29.10.2003 Virtual Organization Purpose –Shared data and computational power for flood forecasting –Cooperation between users for flood forecasting Requirements –Identify and define clear relationships between users –Authentication: certificate authorities –Authorization: access right for each data/resources –Collaborative tools –Security

22. Cracow Grid Workshop ’03, 27-29.10.2003 Flood Forecasting VO Storage systems databases surface automatic meteorological and hydrological stations systems for acquisition and processing of satellite information meteorological radars External sources of information  Global and regional centers GTS  EUMETSAT and NOAA  Hydrological services of other countries Data sources meteorological models hydrological models hydraulic models High performance computers Grid infrastructure Flood crisis teams  meteorologists  hydrologists  hydraulic engineers Users  river authorities  energy  insurance companies  navigation  media  public

23. Cracow Grid Workshop ’03, 27-29.10.2003 Virtual Organization for Fire Simulation Storage systems databases GIS Fuel type - vegetation - canopy cover GIS Topography -elevation -slope Meteorological data source Weather - wind direction, speed - temperature, rel. humidity FIRE MODEL Descriptive Numerical Parameters Data sources Grid infrastructure Users Fire Management Creation Decision Support system, prevention -Terrain, resources, - capacities Fire suppression authorities - training - operation mode Ecosystem authoritiesUniversities, Insurance companies High performance computers Fire Modelling System FARSITE

24. Cracow Grid Workshop ’03, 27-29.10.2003 Data management Typical data: satellite images, radar images, measured data from hydrological stations, topographical data, historical data, simulation results Different formats, different quality, different owners, different access right Metadata server:data description, security, replication

25. Cracow Grid Workshop ’03, 27-29.10.2003 FloodVO data transfer Users High performance computers Data sources Storage systems Databases surface automatic meteorological and hydrological stations systems for acquisition and processing of satellite information meteorological radars External sources of information  Global and regional centers GTS  EUMETSAT and NOAA  Hydrological services of other countries meteorological models hydrological models hydraulic models Grid infrastructure Flood crisis teams  meteorologists  hydrologists  hydraulic engineers  river authorities  energy  insurance companies  navigation  media  public

26. Cracow Grid Workshop ’03, 27-29.10.2003 DataGrid EDG Replica Manager EDG Local Replica Catalogue EDG Replication Metadata Catalogue EDG Replica Optimization Service

27. Cracow Grid Workshop ’03, 27-29.10.2003 Storage Element Storage control Metadata EDG RM EDG RMC EDG ROS EDG LRC DataGrid (cont.)

28. Cracow Grid Workshop ’03, 27-29.10.2003 Grid computing Many multidisciplinary simulations are needed for flood forecasting For critical situations, short response times are very important Numerical simulations are computationally intensive Grid can offer the necessary computational power

29. Cracow Grid Workshop ’03, 27-29.10.2003 Visualization Data are stored in many different formats Unified visualization tools may simplify the user- interface Many data for flood forecasting has spatial character => GIS software may be used as the unified visualization tool

30. Cracow Grid Workshop ’03, 27-29.10.2003 Portal The unified user-interface Allow users access to the VO remotely Simple requirements on clients - based on standard Web technologies

31. Cracow Grid Workshop ’03, 27-29.10.2003 3 current portals 1.Based on GridPort using Globus grid toolkit 2.Based on Jetspeed portal framework using DataGrid/CrossGrid services 3.Migrating Desktop - java fat client using DataGrid/CrossGrid services

32. Cracow Grid Workshop ’03, 27-29.10.2003 GridPort A set of Perl scripts that enable Perl based portal (its CGI scripts) to use grid services of underlying Globus toolkit Wraps Globus’ command line tools Provides session management Provides no additional portal infrastructure

33. Cracow Grid Workshop ’03, 27-29.10.2003 Architecture of GridPort based portal Portal (Apache web server) Storage GridPort toolkit Globus toolkit (GSI, MDS, JobManager, GridFTP, …) Resource 1 Resource 2 Resource n … Storage & Portal Machine User’s web browser …

34. Cracow Grid Workshop ’03, 27-29.10.2003 GridPort screenshot

35. Cracow Grid Workshop ’03, 27-29.10.2003 Job submission in Flood-VO XML file (parameter description) Config. file (default values of parameters) New config file Job script file globus-job-submit machine job_script config_file

36. Cracow Grid Workshop ’03, 27-29.10.2003 Flood-VO: Job list

37. Cracow Grid Workshop ’03, 27-29.10.2003 Flood-VO: Field data II SAS SHMI RDBMS TimeValue 00:00:00102.00000 cm 01:00:00126.00000 cm 02:00:00103.00000 cm 03:00:0080.00000 cm 04:00:0070.00000 cm 05:00:0065.00000 cm

38. Cracow Grid Workshop ’03, 27-29.10.2003 Jetspeed Portal framework Server-side Java based engine (application server) Client services are plugged using software components called portlets. User can arrange portlets – position, size, visibility

39. Cracow Grid Workshop ’03, 27-29.10.2003 Jetspeed - architecture

40. Cracow Grid Workshop ’03, 27-29.10.2003 Application portal screenshot (Jetspeed)

41. Cracow Grid Workshop ’03, 27-29.10.2003 Application Portal

42. Cracow Grid Workshop ’03, 27-29.10.2003 Migrating Desktop (MD) Java application (applet) running at the client computer Provides interface to all basic grid services (authentication, job management, file management) Application specific job parameter input and job submission is supported via application plug-ins has built-in viewer for common picture formats (jpeg, gif, png) and text files, advanced visualization of results via application specific visualization plug-in Being developed in the context of the CrossGrid project

43. Cracow Grid Workshop ’03, 27-29.10.2003 Migrating Desktop Screenshot of MD with Job submission wizard dialog

44. Cracow Grid Workshop ’03, 27-29.10.2003 Use case: Cascade simulation Data sources Hydrological simulation Hydraulic simulation Portal Meteorological simulation

45. Cracow Grid Workshop ’03, 27-29.10.2003 Model characteristics ALADIN (meteorological model) –Limited area model –Operated by 13 Euro-Mediterranean countries –ALADIN/SLOVAKIA operated by SHMI –More than 1M lines of source code (mainly F90) –Developed for 64 bit big-endian architecture –Proprietary - requires nondisclosure agreement

46. Cracow Grid Workshop ’03, 27-29.10.2003 Model characteristics ALADIN (meteorological model) –Type: MPI parallel task, possible parameter studies – multiple executions –CPU time: approximately one hour on 8 processors –I/O size: 33/180 MB per run –Scalability: on fast Ethernet up to 8 processors –Input data: boundary conditions –Output data: quantitative precipitation forecast, temperature

47. Cracow Grid Workshop ’03, 27-29.10.2003 Model characteristics HSPF (hydrological model) –Type: sequential task, multiple executions (high throughput computing) –CPU time: very small (seconds - minute) –I/O size: 1-10 MB –Scalability: HTC –Input data: quantitative precipitation, temperature, topographical data –Output data: hydrograph

48. Cracow Grid Workshop ’03, 27-29.10.2003 Model characteristics FESWMS (hydraulic model) –Funded by US Federal Highway Administration –Distributed in commercial package SMS by EMS-I –Source code available (direct cooperation with developer) –Optimized and parallelized by II SAS

49. Cracow Grid Workshop ’03, 27-29.10.2003 Model characteristics FESWMS (hydraulic model) –Type: MPI parallel task, multiple executions with different input data –CPU time: 10min to several hours per a task –I/O size: 10-100 MB –Scalability: good for smaller number of processor (to 16). –Input data: inflow, topographical data –Output data: water levels and velocities

50. Cracow Grid Workshop ’03, 27-29.10.2003 DaveF model A time-explicit finite-volume model from the same developers as FESWMS. It is considered as the complement of FESWMS and it is best suitable for unsteady state with critical or super-critical flow (dam- breaking, flash flood, flood with wetting/drying in large expanses) DaveF uses the same graphical environment like FESWMS (SMS) and similar input/output format =>can be easily added into ANFAS system Parallel version of DaveF has been developed for clusters by II-SAS and shows good results

51. Cracow Grid Workshop ’03, 27-29.10.2003 Architecture Portal Computing element Resource broker Storage element

52. Cracow Grid Workshop ’03, 27-29.10.2003 Login to FloodVO Portal Computing element Resource broker Storage element

53. Cracow Grid Workshop ’03, 27-29.10.2003 Choose simulation Portal Computing element Resource broker Storage element

54. Cracow Grid Workshop ’03, 27-29.10.2003 Enter input parameters Portal Computing element Resource broker Storage element

55. Cracow Grid Workshop ’03, 27-29.10.2003 Visualization Portal Computing element Resource broker Storage element

56. Cracow Grid Workshop ’03, 27-29.10.2003 Visualization Portal Computing element Resource broker Storage element

57. Cracow Grid Workshop ’03, 27-29.10.2003 Download simulation results Portal Computing element Resource broker Storage element

58. Cracow Grid Workshop ’03, 27-29.10.2003 Cascade simulation Portal Computing element Resource broker Storage element Call meteorology master script Run meteorological simulation Extract hydrological input from results Call hydrology master script Run hydrological simulation Check results if (inflow > critical flow) If yes: call hydraulics master script run hydraulic simulation

59. Cracow Grid Workshop ’03, 27-29.10.2003 Cascade simulation Portal Computing element Resource broker Storage element Resource broker Computing element Computing element Storage element Portal Resource broker Storage element Call meteorology master script Run meteorological simulation Extract hydrological input from results Call hydrology master script Run hydrological simulation Check results if (inflow > critical flow) If yes: call hydraulics master script run hydraulic simulation

60. Cracow Grid Workshop ’03, 27-29.10.2003 Cascade simulation Portal Computing element Resource broker Storage element Call meteorology master script Run meteorological simulation Extract hydrological input from results Call hydrology master script Run hydrological simulation Check results if (inflow > critical flow) If yes: call hydraulics master script run hydraulic simulation

61. Cracow Grid Workshop ’03, 27-29.10.2003 Cascade simulation Portal Computing element Resource broker Storage element Call meteorology master script Run meteorological simulation Extract hydrological input from results Call hydrology master script Run hydrological simulation Check results if (inflow > critical flow) If yes: call hydraulics master script run hydraulic simulation

62. Cracow Grid Workshop ’03, 27-29.10.2003 Cascade simulation Portal Computing element Resource broker Storage element Resource broker Computing element Computing element Storage element Portal Resource broker Storage element Call meteorology master script Run meteorological simulation Extract hydrological input from results Call hydrology master script Run hydrological simulation Check results if (inflow > critical flow) If yes: call hydraulics master script run hydraulic simulation

63. Cracow Grid Workshop ’03, 27-29.10.2003 Cascade simulation Portal Computing element Resource broker Storage element Call meteorology master script Run meteorological simulation Extract hydrological input from results Call hydrology master script Run hydrological simulation Check results if (inflow > critical flow) If yes: call hydraulics master script run hydraulic simulation

64. Cracow Grid Workshop ’03, 27-29.10.2003 Cascade simulation Portal Computing element Resource broker Storage element Call meteorology master script Run meteorological simulation Extract hydrological input from results Call hydrology master script Run hydrological simulation Check results if (inflow > critical flow) If yes: call hydraulics master script run hydraulic simulation

65. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=315 minutes

66. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=510 minutes

67. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=720 minutes

68. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=810 minutes

69. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=915 minutes

70. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=1005 minutes

71. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=1110 minutes

72. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=1305 minutes

73. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=1515 minutes

74. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=1710 minutes

75. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=1905 minutes the maximum water level

76. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=2100 minutes

77. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=2310 minutes

78. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=2700 minutes

79. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation for t=2910 minutes

80. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation step 1time 0

81. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation step 3 time 0:30

82. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation step 4 time 0:45

83. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation step 50 time 12:15

84. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation step 100 time 24:45

85. Cracow Grid Workshop ’03, 27-29.10.2003 Dynamic flood simulation step 150 time 37:15

86. Cracow Grid Workshop ’03, 27-29.10.2003 Future work Adding more models Performance analysis and optimization Improving data management (repositories) Adding more information about data (metadata) Adding collaborative tools

87. Cracow Grid Workshop ’03, 27-29.10.2003 Use case: Simulation sequence Hydrology Hydraulics Meteorology CZ Meteorology SK Meteorology A Meteorology D Meteorology H Meteorology CH

88. Cracow Grid Workshop ’03, 27-29.10.2003 Danube River basin Experts Data providers Navigation Computing centers Flood crisis teams River authorities Media Public Virtual Organization for Flood Forecasting FloodGrid Energy Insurance companies simulations management information

89. Cracow Grid Workshop ’03, 27-29.10.2003 Knowledge based Flood forecasting Data SourcesVisualization/Output Processing Meteorological Simulations Hydrological Simulations Hydraulic Simulations Metadata & Knowledge Repository Automatic or Semiautomatic Reasoning External Resources (Web services) Grid infrastructure Simulation Metadata (parameters, area, sim. method, etc) User interaction (Semiautomatic) Job Submission (based on evaluation of previous jobs outputs) Information about job run (relevance) Actual data from observation stations (relevance)