1. SWAT gridification short summary
Łukasz Kokoszkiewicz

2. Contents EnviroGRIDS project overview SWAT gridification assumptions
SWAT explanation
SWAT gridification assumptions
Parallelization results
SWAT model parallelization
LH-OAT uncertainty analysis parallelization
SWAT-CUP sufi2 parallelization
EnviroGRIDS VO
Monitoring Tools

3. Partners involved in grid computing in the project are circled in RED
DHMO
Fortunately we have a large range of skills and expertise among the 27 partners of the project that all have particular tasks to fulfill in the project.
Partners involved in grid computing in the project are circled in RED
Project timeframe: April 2009 – March 2013
Total budget of €8.1 mio. (€6.2 mio. EC contribution) and 1367 man-months.
The EnviroGRIDS Project Team includes 27 partners from 15 countries, representing also several European (CERN, EEA) and United Nations organizations (UNEP, UNESCO).
Among these partners, 22 belong partially or entirely to the Black Sea Catchment.
Eight partners belong to an International Cooperation Partner Countries (Ukraine, Georgia and Russian Federation) and five to Associated Countries (Switzerland and Turkey).
Five partners can be defined as Small and Medium Enterprises (SME).

4. technical objectives data gap analysis
modeling of large scale, high resolution distributed hydrologic processes
developing a grid-enabled Spatial Data Infrastructure (SDI)
spatially explicit regional scenarios of development
develop access to real time data from sensors and satellites
develop early warning and decision support tools at regional, national and local levels
Build capacities in the Black Sea region to enhance data interoperability, by promoting on OGC and open standards (ISO 19115/19119/19139; web services WMS, WFS, WCS)
to run a gap analysis on existing regional observation systems to prepare recommendations for improvement of networks of data acquisition in each region/country,
Create regional scenarios of development in function of expected climate, land cover and demographic changes
to gridify the software that compute large scale resolution modeling of distributed hydrologic processes,
to develop the access to real time data from sensors and satellites,
to streamline the production of indicators on sustainability and vulnerability of societal benefits. Streamline data process from data warehouses, to scenarios, hydrological models, impacts assessments and finally to disseminations tools.
to provide policy-makers and citizens with early warning and decision support tools at regional, national and local levels.
to build capacities in the implementation of many new standards and frameworks (INSPIRE, GEOSS,..).

5. SWAT
SWAT (Soil and Water Assessment Tool) is a river basin, or watershed, scale model.
It predicts the impact of land management practices on water, sediment and agricultural chemical yields in large complex watersheds with varying soils, land use and management conditions over long periods of time.
- SWAT widely is used around the world by entire hydrology community.
SWAT and related software is constantly developed in many institutions around the world
SWAT was developed since the early 1990s.
- SWAT will the model every part of an hydrological model across the full landscape under study.
- The main input of water is precipitation, which divides itself into Surface Runoff, Lateral flow, Return Flow, Recharge of Aquifer and of course evaporation.

6. swat
Models
We already used SWAT on Lake Balaton in Hungary and many users are using this tool around the world to study their river catchments. In order to run this model, we need relatively simple data:
Altitude and slope
Land cover
Soil characteristics
Climatic time series of temperature and rainfall data from weather stations
Then, in order to calibrate the model we need hydrological data to fit the model to the observations

7. Hydrological modeling
This is visualisation of hydrological model of Balaton Lake.
Blue lines are rivers
Blue dots are rivers inlets
Red lines are the borders of sub-basins used in calculating predictions for the hydrological model.

8. gridification assumptions
SWAT Gridification
Creating EnviroGRIDS VO
Research which use cases are possible to utilize on the GRID
Creating easy to use monitoring tools for the community
Envirigrids VO exists for few months now
Easy to use client application called hBrowse framework was created and based on it Diane and Ganga monitoring tools were created
3 use cases, for the SWAT gridification, were found

9. SWAT MODEL gridification
Sub-basin based model splitting followed by routing
The main idea is to split model itself based on sub-basins.
Pre and post processing of the model is required to complete splitting and merging steps.
After splitting, each sub-model can be run separately (in parallel) on the grid.
Subbasin based parallelization is very interesting for large models.
Mention:
Splitting onto subbasns
Run each subbasin on different node
Merging results into full model
Conclusions
Not proven to be useful
Maybe for a really big models with many HRU’s

10. Danube river 15K HRU's model grid runs
As was observed SWAT model sub-basin based grid executions show some promises but to be fully utilized it requires much bigger jobs. Ideal situation would be when single sub-basin execution time takes more than half an hour, in this kind of situations the grid overhead would be negligible.
Time [s]
Example chart showing execution times in a function of simulation length
- Flat red line shows grid execution
- Blue line is the non-parallel run
- At the beginning we can see that parallelisation operations and grid overhead makes the execution slightly slower for a very short simulation time
Simulation length [years]

11. LH-OAT sensitivity analysis alghoritm parallelization
LH-OAT is an uncertainty analysis method.
In uncertainty analysis we are sorting out parameters sensitive to selected real world physical process (eg. nitrogen transport).
Then an effective model calibration can be applied.
The mechanism behind LH-OAT method parallel run is very similar to SWAT-CUP parallelization.
We simply run many SWAT runs with a different parameters.
SWAT Model
LH-OAT Splitter
GRID
SWAT Model, params set 1
SWAT Model, params set 2
SWAT Model, params set n
swat
swat
swat
Latin Hypercube One factor At a Time
Not properly tested yet
It require some rethinking in a splitter section
Not proven to be useful yet
Output
Output
Output
LH-OAT Merger

12. SWAT-CUP sufi2 algorithm parallelization
SWAT-CUP is an application for calibration of SWAT models.
Calibration of SWAT models requires running SWAT simulations hundreds of times with different parameters set
The idea here is to run them separately or in small independent groups and resolve the best parameter configuration that gives simulation effects most corresponding to reality
We have “ready to go” example script set, to run swat sufi2 calibration efficiently from command line
GRID
Parameters
New SWAT Inputs
swEdit.exe
MCMC
swat
SUFI-2
ParaSol
SWAT Outputs
SWAT-CUP sufi2 gridification brings the greatest promises to the SWAT gridification
Gridified code proven to be most performance effective in comparison to non-parallelized one
In theory, it allows to run calibration in a time close to time needed to run single SWAT simulation, and giving that calibration can take easily eg. 500 loops the time savings become very significant
GLUE
swExtract.exe
Output
SWAT-CUP algorithms

13. Envirogrids vo EnviroGRIDS VO is up and running since Autumn 2010
Resources from Romania were added shortly after the VO were started and are being used since
CE - ce01.mosigrid.utcluj.ro:8443/cream-pbs-envir – 960CPUs
SE - se01.mosigrid.utcluj.ro terabytes
Registration
Approval is automatic for a listed members of the EG project
People that are not members should be “recommended” by a listed member
VO configuration
voms113.cern.ch needs to be replaced by voms.cern.ch inside the VOMSES string
CIC VO Identity card:
EnviroGRIDS VO is up and running
For now, only resources from Romania were added into pool
As I’m the VO manager for now, I’m the one who approves new members
So my policy is the following
If one doesn’t appear on the official EG website then someone who does needs to “recommend” the user (totally informal, just send me an from your official EG mailbox)
Listed EG members are automatically accepted
Some configuration facts

14. Monitoring hBrowse framework
hBrowse.net
It’s a common, easy to configure, client side, tool for creating job monitoring systems
It can visualise any hierarchical data up to 4 levels deep in a neat, table based environment
Google charts and highcharts can serve as renderer in creating visually attractive charts
Based on hBrowse framewrok many monitoring tools was created including:
The client can be configured in just one file
Quite a success, used by many communities (basically can be used by everyone that used ganga and diane)

15. Hbrowse System architecture
Monitoring
Hbrowse System architecture
The main system component is a JavaScript client which serves as a presentation layer.
Different server layers can be applied.
Server layer serves only as a data source.
Client and server communicates using JSON data transfer format.
JSON in this case is a perfect solution because of its syntax:
in most cases it is the same as in JavaScript and Python
CLIENT
JavaScript
Ajax
jQuery
JSON
HTTP REQUEST
hBrowse framework is only the client side application, if one wants to used he have to provide his own server part that serves the data in the json format
SERVER
DATABASE
WEB SERVER
ActiveMQ messages

20. Final thoughts SWAT Gridification state
The state of the SWAT gridification is that we can run swat tasks on the grid
With monitoring tools we can easily monitor our simulations/calibrations
SWAT Gridifications final requirements
Support of the linux executables (preferably both 32-bit and 64-bit) from SWAT developers is critically required
Very important is to start official support for the linux swat executables!!!
Officially certified executables are needed to ensure proper output data in all organisations that would want to use it

21. Thank you