Rapid Raster Projection Transformation and Web Service Using High-performance Computing Technology 2009 AAG Annual Meeting Las Vegas, NV March 25th, 2009 Qingfeng (Gene) Guan Michael P. Finn E. Lynn Usery David M. Mattli Center of Excellence for Geospatial Information Science U.S. Geological Survey Rolla, MO

Contents Motivations Parallelizing raster projection transformation Static load-balancing Dynamic load-balancing pRPL – parallel Raster Processing programming Library Conclusions

Motivations Massive data volumes Re-sampling method Example: High resolutions Large areas Easily 500MB+ Re-sampling method Multiple projection transformations for each output pixel (center and corners) Example: Global land-cover at 30-sec resolution, 21,600X43,200 pixels, 900 MB Geographic → Mollweide Laptop, Intel Pentinum M 1.5 GHz, 1.25 GB RAM 45 minutes 10 seconds!!

Inverse Projection Transformation

Motivations Problem: High computational intensity v.s. demand for rapid projection (web) service Solution: High-performance computing technologies Parallel computing

Parallel approach for raster data Raster is born to be parallelized A raster dataset is essentially a matrix of values, each of which represents the attribute of the corresponding cell of the field A matrix can be easily partitioned into sub-matrices and assigned onto multiple processors so that the sub-matrices can be processed simultaneously

Parallelizing Projection Transformation Output image is decomposed Minimum Bounding Rectangles (MBRs) of sub-input-images are computed using the MBRs of sub-output-images

Parallelizing Projection Transformation Static Load-balancing

Parallelizing Projection Transformation Dynamic Load-balancing Reduced granularity Master reads sub-input-images and distributes them in response to requests Workers/Slavers request for new tasks (MBRs of sub-output-images & corresponding sub-input-images)

Parallelizing Projection Transformation Dynamic Load-balancing

pRPL: parallel Raster Processing Library An open-source general-purpose parallel Raster Processing programming Library Encapsulates complex parallel computing utilities and routines specifically for raster processing Enables the implementation of parallel raster-processing algorithms without requiring a deep understanding of parallel computing and programming Possible usage Massive-volume geographic raster processing Image (including remote sensing imagery) processing Cellular Automata (CA) and Agent-based Modeling (ABM) Freely downloadable and open source http://sourceforge.net/projects/prpl/

pRPL (cont.) Object-Oriented programming style Written in C++ Built upon the Message Passing Interface (MPI) Provides Transparent Parallelism Supports almost all types of raster-based processing Local-scope Neighborhood-scope Regional-scope Global-scope

pRPL 2.0 – under development Supports dynamic load-balancing for data parallelism Master-worker formation Master Reads data dynamnically Distributes the initial subsets to the workers Maintains the task farm which contains the remaining subsets of data Sends the subsets to the workers in respond to requests Receives completed output subsets from workers Workers/Slavers Initially assigned with some subsets of data Request for more data when finish the assigned subsets Receive new input subsets to process from the master Submit completed output subsets to the master

Conclusions Massive-volume raster projection transformation needs high-performance computing technology Dynamic load-balancing technique improves performance Reduces the I/O overhead and the requirement for memery space Improves the utilitization rate (efficiency) of a heterogenerous parallel computing system pRPL reduces the devolopment complexity of a parallel raster-based processing

Thank You! Questions & Comments?

Your Topic Goes Here Your Subtopics Go Here

Your Topic Goes Here Your Subtopics Go Here

elements www.animationfactory.com