Developing an Agricultural Monitoring System from Remote Sensing Data Using GridRPC on Ninf-G Shamim Akther, Yann Chemin, Honda Kiyoshi Asian Institute of Technology, Thailand Itthichok Jangjaimon, Putchong Uthayopas Kasetsart University, Thailand And Thai National Grid Project

2 Motivation Recent advances in data assimilation of satellite imagery for the purpose of agricultural monitoring led to the increasing demand for computing power There is a need for technology and platform that can meet the requirement

3 SWAP Model and Structure Adopted from Van Dam et al. (1997) Drawn by Teerayut Horanont (AIT)

4 SWAP-GA Know SWAP Input parameter – sowing date, soil property, Water management, etc Finding SWAP Crop model

5 SWAP Model Parameter identification - Data Assimilation using RS and GA - Data Assimilation using RS and GA Day Of Year Evapotranspiration LAI RS Observation SWAP Crop Growth Model SWAP Input Parameters sowing date, soil property, Water management, and etc. LAI, Evapotranspiration Day Of Year Eavpotranspiration LAI Fitting LAI, Evapotranspiration Assimilation by finding Optimized parameters By GA RS Model

6 Pixel poolResult pool Best fit population Sequential Version Take satellite image and loop through each pixel For each pixel perform GA processing

7 Challenges The calculation time for identify SWAP parameters only for 1 pixel (1 sq.km) takes several minutes to 30 minutes. Thus, a RS image of 1000 x 1000 sq.km of 1000x1000 pixels will take more than 50 years (30min x 1000 x 1000) is not acceptable. Solutions –Parallel Computing on Cluster and Grid Longitude: Latitude:

8 Thai Grid Infrastructure

9 Programming Model Message Passing Using MPI –Widely adopted standard –Work efficiently on Cluster –Can work at some level on Grid Grid RPC –Remote procedure call programming model on Grid system –GGF adopted as a standard programming on Grid –Several Implementation : GridSolve, Ninf-G

10 GridRPC Programming Model Application partition into client/server –Multiple client can be used to achieve parallelism using asynchronous call Role –Server side Implement the algorithm as a remote procedure Using Grid for security mechanism and execution management –Client side Implement main workflow of the application Handling input/output for user Call Remote Functions at Server using GridRPC API

11 Ninf-G programming model A software package which supports programming and execution of Grid applications using GridRPC Advantages –Simple programming model that work efficiently over lossely copuple heterogenous system –Hiding most of the grid complexity from programmer and developer Low level Grid Middleware Grid RPC Grid application Grid Fabric Layer

12 Partitioning strategy on clusters and Grid There are 3 possible partitioning strategies –Population Distribution –Pixel Distribution –Hybrid Distribution Group compute nodes as virtual group each virtual group take care one pixel inside the group, distributing population to members Pixel and population distributed has been implemented using MPI and reported in our previous work

13 Population Distribution Strategy Pixel poolResult pool Eval GA Set of Population fitness

14 Pixel Distribution Strategy Pixel poolResult pool GA Image Pixel Results

15 Hybrid Distribution Strategy GA Pixel poolResult pool Image Pixel Results Eval population Cluster GA Pixel Results Eval population Cluster

16 Extension to the Grid Modify and adjust code to change distributing middleware from MPI to Ninf-G Implement secure data-transferring method for Grid instead of using NFS for cluster environment Gain benefit of Grid computing –More effective –More scalable –More security

17 Architecture

18 GA_SWAP(pixel_buffer, result) { Load(pixel_buffer); Run(swap); Send(result); }

19 Results & Discussions Finish only pixel distribution over cluster Experiment run on 3 clusters in the Grid –MAEKA 31 nodes Opteron Dual 1.4&1.8 GHz –AMATA 14 nodes Athlon XP 1 GHz –MAGI 4 Nodes Athlon XP 2500+

20 Result

21 Result

22 Result

23 Lesson Learned Grid programming is still difficult –Learning curve –Debugging? –Grid reliability. May be useful to have a small development grid separate from deployment grid Ninf-g application on grid is working fine. A good approach for development of grid applications.

24 Future Work Improve performance scheduler and node selecting (now is only round-robin) Test running across international Grid –collaboration with Prof. Kento Aida,Titech –PRAGMA testbed Deployment of ninf-g for Thai National Grid application development

Thank you Q&A