Opportunities for SLEUTH in High-performance Computing Qingfeng (Gene) Guan, Ph.D. Center for Advanced Land Management Information Technologies School of Natural Resources University of Nebraska - Lincoln 2102 AAG Annual Meeting New York, NY

Content Introduction pRPL – a general-purpose Raster Processing programming Library pSLEUTH – parallelizing SLEUTH using pRPL Parallelizing CA using GPUs Conclusions

Introduction - SLEUTH The urban growth model SLEUTH, uses a modified Cellular Automata to model the spread of urbanization across a landscape (Clarke et al., 1996, 1997). Its name comes from the GIS data layers that are incorporated into the model; Slope, Landuse, Exclusion layer (where growth cannot occur, like the ocean), Urban, Transportation, and Hillshade. (Goldstein. 2004) Prediction of urban development to the year 2050 over southeastern Pennsylvania and part of Delaware using the SLEUTH model mation/movie_small.htm

Coefficients – Dispersion – Breed – Spread – Slope – Road Gravity Rules – Spontaneous Growth Rule (centralized) – New Spreading Centers Rule (non-centralized) – Edge Growth Rule (non- centralized) – Road-Influenced Growth Rule (non-centralized) For more info. about SLEUTH: Introduction - SLEUTH

Bruce Force Calibration – coefficient combinations Monte Carlo Iteration – 10 ~ 100 Iterations for each coefficient combination Massive-volume Data – High spatial resolution – Large study area High Computational Intensity –The model calibration for a medium sized data set and minimal data layers requires about 1200 CPU hours on a typical workstation (Clarke. 2003) Calibration of SLUETH Introduction - SLEUTH

Solution 1: Simplifying Assumptions – Each coefficient affects the model behavior in a linear manner – Iteratively narrow down the coefficient search range 0, 50, , 75, , 62, 75 … Reach the optimal coefficient value – Problem: random processes in CA very likely non-linear relationships assumptions generate unreliable calibration results inaccurate simulation results possibly flawed conclusions and theories, and improper management and planning decisions

Introduction - SLEUTH Solution 2: Smart Agents – Computational Intelligence – Genetic algorithm, Artificial Neural Networks, Ant Intelligence, etc. – Search the coefficient space more efficiently – Issue: which one is better/smarter? Solution 3: High-performance Computing (HPC) – Dumb Dumb with Fast Legs – Exhaust the coefficient space in a acceptable/feasible time period – Remove simplifying assumptions

Introduction – High-performance Computing (HPC) Parallel computing is the use of multiple computing units (computers, processors, or processes) working together on a common task in a concurrent manner in order to achieve higher performance In contrast to sequential computing that usually has only one computing unit Performance is usually measured with computing time A massive parallel computing system ( )

pRPL: parallel Raster Processing Library Raster is born to be parallelized

Writing a parallel raster processing program is as simple as writing a serial program Possible usage – Massive-volume geographic raster processing – Image (including remote sensing imagery) processing – Large-scale Cellular Automata (CA) and Agent-based Modeling Free downloadable and open source – pRPL - Introduction

Object-Oriented programming style – Written in C++ – Built upon the Message Passing Interface (MPI) Class templates support arbitrary data types – e.g. integer, char, double precision floating point number, even user-defined types Transparent Parallelism pRPL - Features

Spatially Flexible – Supports any arbitrary neighborhood configuration – Supports centralized and non-centralized algorithms pRPL - Features (cont.)

Regular and irregular decomposition pRPL - Features (cont.)

Multi-layer processing pRPL - Features (cont.)

The four growth rules in the SLEUTH model, were implemented using pRPL pSLEUTH - Parallelization

Processer Grouping – With pRPL, pSLEUTH is able to organize the processors in groups. Data parallelism within a group, task parallelism among groups. – Static tasking and dynamic tasking pSLEUTH - Parallelization (cont.)

Data – Urban areas of the continental US (1980 and 1990, 4948×3108) Calibration Settings – Only three values (0, 50, 100) will be evaluated for each coefficient – The total number of simulations is 243 (= 3 5 ) – Each simulation includes 11 ( = ) years pSLEUTH - Experiments Settings

Computer cluster at University of California, Santa Barbara – 128 dual CPU 3.06 GHz Xeon nodes – 2-GB RAM each

pSLEUTH – Performance

GPU - Introduction Graphics Processing Unit (GPU) – Massive number (hundreds) of computing cores on one unit – Very inexpensive ($200 - $2000) – General parallel processing with Compute Unified Device Architecture (CUDA) – Personal Supercomputer

GPU – Preliminary Study Classical CA - Game Of Life (GOL) – 10,000 X 10,000 – 100 iterations Desktop PC with 1.6 GHz dual-core CPU – 100 minutes Desktop PC with NVIDIA GeForce GTX 260 GPU – 192 cores – 6 minutes – 16.6 speed-up

GPU – Preliminary Study Keeneland – Hybrid computer cluster with multiple CPUs and GPUs – 120 nodes with 240 CPUs and 360 GPUs – Developed by Georgia Institute of Technology, University of Tennessee at Knoxville, and Oak Ridge National Laboratory, sponsored by NSF Keeneland-GOL – 8 GPUs used, 25 seconds, 240 speed-up – 20 GPUs used, 20 seconds, 300 speed-up

HPC greatly reduces the computing time for model calibration – Enables the removal of simplifying assumptions – Can increase the reliability of calibration and accuracy of simulation New computing accelerator technology enables HPC-SLUETH with very low costs Opportunities for us to test and verify conclusions and theories, as well as to stimulate new theories Conclusion

Comments and Questions?