Towards GPU-Accelerated Web-GIS Jianting Zhang1,2, Simin You2,4, Le Gruenwald3 1The City College of New York 2CUNY Graduate Center 3University of Oklahoma 4Pitney Bowes Inc.

Outline Background & Motivation System Design and Prototyping On-Demand Data Parallel Spatial Joins on GPUs WebGIS Frontend Optimized for QDVE Application Case, Experiments and Results Conclusion and Future Work

Introduction & Background GIS Vis Web-GIS and Parallel and Distributed Processing Spatial Joins on GPUs Previous Efforts on WebGIS for QDVE on Large-Scale Geospatial Data Other Related Works Web Web-GIS Big Data and HPC

Parallel Spatial Join Framework SELECT * from T1, T2 WHERE ST_OP (T1.the_geom, T2.the_geom) Spatial Data Processing Spatial Data Management (Spatial Databases) Spatial Indexing Spatial Join Filtering phase Refinement phases Point quadrant Polyline/polygon MBB Spatial indexing on quadrants of points or MBBs (Minimum Bounding Boxes) of polyline/polygons Spatial join includes the filtering phase and refinement phase Spatial filtering uses spatial indices to match pairs of MBBs in T1 and T2 based on spatial intersection (overlapping) and reduce complexity from O(n1*n2) to O(n1) or O(n2) Spatial refinement performs geometrical operations (e.g., point-to-polyline distance or point-to-polygon test) on matched pairs Many spatial indexing and spatial join techniques have been proposed in the past few decades.

Introduction & Background Jianting Zhang, Michael Gertz, Le Gruenwald: Efficiently managing large-scale raster species distribution data in PostgreSQL. ACM-GIS 2009: 316-325 ACM-GIS’09 Postgresql+LTree Decompose complex polygons into linear quadtree nodes Convert spatial query into string matching Support all traditional WebGIS systems 4000+NaturalServe Birds Species range maps 6-9.5X speedups than Postgresql+PostGIS Data structures and Preprocessing matters!

Introduction & Background Jianting Zhang: A high-performance web-based information system for publishing large-scale species range maps in support of biodiversity studies. Ecological Informatics 8: 68-77 (2012) Using same idea on decomposing polygons in to linear quadtree nodes Maintain the tree in main-memory for fast and efficient point/range query processing Frontend: OpenLayers Response time < 1s Suitable for simple interactive query in a WebGIS

Introduction & Background Jianting Zhang, Simin You: Supporting Web-Based Visual Exploration of Large-Scale Raster Geospatial Data Using Binned Min-Max Quadtree. SSDBM 2010: 379-396 Jianting Zhang, Simin You: Dynamic tiled map services: supporting query-based visualization of large-scale raster geospatial data. COM.Geo 2010 Also main-memory based Binned Min-Max Quadtree for rasters Frontend: ArcGIS Flex API Provides dynamic tile services for highlighting query results

Introduction & Background Jianting Zhang, Simin You, Yinglong Xia: Prototyping A Web-based High-Performance Visual Analytics Platform for Origin-Destination Data: A Case study of NYC Taxi Trip Records. UrbanGIS@SIGSPATIAL 2015: 16-23 Frontend: Google Map Backend: multi-core CPU processing based on OpenMP Interactive response for user-defined simple polygons

7.1 billion transistors (551mm²) 2,688 processors ASCI Red: 1997 First 1 Teraflops (sustained) system with 9298 Intel Pentium II Xeon processors (in 72 Cabinets) Feb. 2013 7.1 billion transistors (551mm²) 2,688 processors 4.5 TFLOPS SP and 1.3 TFLOPS DP Max bandwidth 288.4 GB/s PCI-E peripheral device 250 W (17.98 GFLOPS/W -SP) Suggested retail price: $999 This slides shows ASCI Red, the world’s first supercomputer with Teraflops double precision computing power in 1997 and the Nvidia GTX Titan available from the Market in 2013 for $1000 with even more computing power. Now the question is: Many years ago, some database researchers tried to develop specialized hardware for database applications. However, that did not generate the expected impacts from the industry due to high cost. Personally, I do not believe GPUs are perfect for data management. However, I do believe that it is the most cost-effective hardware for high-performance data management, specially for geospatial data which is both data intensive and computing intensive. The simple belief has motivated us to work continuously over the past four years and we are happy to receive an NSF grant to continue the research in the next four years. To our knowledge, Prof. Kenneth Ross’s group at Columbia is also working on GPU-based data management, but they are mostly focusing on relational data. What can we do today using a device that is more powerful than ASCI Red 17 years ago?

Nvidia P100

Computer Architecture Spatial Data Management How to fill the big gap effectively? David Wentzlaff, “Computer Architecture”, Princeton University Course on Coursea

Data Parallelisms  Parallel Primitives Parallel libraries Parallel hardware Gather Reduction Scatter Map Scan Source: http://parallelbook.com/sites/parallelbook.com/files/SC11_20111113_Intel_McCool_Robison_Reinders.pptx Our GPU-based Spatial Data and Trajectory Data management techniques utilize parallel primitives (map, reduce, sort, scan, gather, scatter, etc.) whereas possible to reduce development complexity, increase productivity and maintain portability

Outline Introduction & Background System Design and Prototyping On-Demand Data Parallel Spatial Joins on GPUs WebGIS Frontend Optimized for QDVE Application Case, Experiments and Results Conclusion and Future Work

Data Parallel Spatial Filtering on GPUs   Data Parallel Spatial Filtering on GPUs WebGIS frontend   Applications Global Biodiversity dataset Taxi Trip dataset at NYC   QDVE Primitives Overview (Aggregation query) Filter and Zoom(spatialnon-spatial) Context+Focus: (ROI query on neighborhood) Details on Demand (Identifying Query)   GUI (Dataset Selection, ROI Selection, Join Selection, Layer Selection) Frontend Geometry Library (MBR Indexing, MBR Intersection Test, Point-in-Polygon Test) Network/Web Communication Javascript asynchronous function call JSON string encoding and parsing User data and Javascript binding  

Data Parallel Spatial Filtering on GPUs based on flat grid-file for Point-in-Polygon test based spatial join Row-major order: for cell at (r,c) cell_id=r*width+c Step1: point indexing Input: a vector of points (x,y) Outputs (1) a vector of grid cells identifiers in row-major order (2) A vector of the numbers of points in each grid cell (3) A vector of positions pointing to the first points in grid cells. (4) A sorted point vector with points within a grid cell close to each other X/Y Coordinates transform stable_sort_by_key 1 2 5 5 Cell-ID 3 3 3 1 2 2 reduce_by_key 3 Len 3 1 2 2 exclusive_scan Offset 3 4 6 4

Data Parallel Spatial Filtering on GPUs based on flat grid-file Step2:polygon MBR indexing W*H Input: a vector of polygon vertices (x,y) , a vector of boundary vertex positions and a vector of #of vertices for polygons Outputs: (similar structures) cell_id 4 7 11 14 cell_len 3 1 2 2 3 4 6 poylgon_id 1 2 3 4 5 6 7 (2,2) 16 17 18 19 23 24 25 26

Data Parallel Spatial Filtering on GPUs based on flat grid-file Step3: cell-matching 11 1 2 4 p_id q_id c_id Vectorized Binary Search to pair point cells and polygon cells q_id 1 2 3 p_id 1 1 p_cell_id q_cell_id 3 4 11 22 4 7 11 14 p_cell_len 3 1 2 2 q_cell_len 3 1 2 2 3 4 6 3 4 6 poylgon_id poylgon_xy poylgon_pos

Data Parallel Spatial Filtering on GPUs based on flat grid-file Step4 (optional) box-in-polygon test based optimization for advanced spatial filtering If at least one sides of a box (cell) intersects with at least one polygon edge, then they intersect and all the points in the cell need to go through point-in-polygon test (regular spatial refinement) Otherwise, pick up a corner, if it is in the polygon, then all points in the cell are within the polygon without going through point-in-polygon tests For all the rest cases, all points in the cell are outside of the polygon and can be safely ignored

Parallelization on GPUs Spatial Refinement (for point-in-polygon test) int pnpoly(int npol, float *xp, float *yp, float x, float y) { int i, j, c = 0; for (i = 0, j = npol-1; i < npol; j = i++) { if ((((yp[i] <= y) && (y < yp[j])) || ((yp[j] <= y) && (y < yp[i]))) && (x < (xp[j] - xp[i]) * (y - yp[i]) / (yp[j] - yp[i]) + xp[i])) c = !c; } return c; CPU Code: 7 lines of C code due to W. Randolph Franklin@RPI Parallelization on GPUs Native CUDA implementation (nested loop) A matched pair (p, q) is assigned to a CUDA Thread Block Each point in the grid performs PIP test with the corresponding polygon from (p,q) pair Polygon vertices Points Perfect coalesced memory accesses Utilizing GPU floating point power

Example cid pid qid status pid 1 2 # of point 4 12 a 14 1 30 2 c 31 3 a 14 1 30 2 c 31 3 42 4 63 5 67 6 68 7 72 8 84 9 88 10 98 11 130 135 149 b 155 15 189 16 202 17 205 18 206 19 207 20 221 21 223 22 234 23 236 24 pid 1 2 # of point 4

Outline Introduction & Background System Design and Prototyping On-Demand Data Parallel Spatial Joins on GPUs WebGIS Frontend Optimized for QDVE Application Case, Experiments and Results Conclusion and Future Work

http://gbif.org Global Biodiversity Data at GBIF Application Case: analyzing how species are distributed on the Earth Pick up a group of species and a zonal dataset, count the numbers of occurrences (points) in zones (polygons) Global Biodiversity Data at GBIF http://gbif.org ~375 million occurrence points (as of 2012) for 1,487,496 species ~15 thousands region (WWF ecoregions) Five groups of species are selected for experiments Hardware (GPU device): 2013 Nvidia GTX Titan GPU with 2,688 cores and 6 GB memory

Server backend performance test results Baseline: libspatialindex+GDAL on a single CPU core processes 131 points per second http://geoteci.engr.ccny.cuny.edu/zs_supplement/zs_gbif.html

Summary and Future Work We have proposed a new WebGIS framework and the techniques to leverage GPU-accelerated spatial join techniques for query driven visual explorations on large-scale geospatial data. The design and implementation considerations are discussed in details and an application case on exploring global biodiversity data is presented. The experiment results have demonstrated the feasibility of the proposed framework and the desired high performance The work is preliminary in nature from a system development perspective Evolving the prototype towards a more mature system that can serve as a generic platform to accommodate more application cases Integrating both CPUs and GPUs to further enhance backend performance Developing novel ideas on designing effective GUI interfaces for complex QDVE workflows Community efforts are invited!

Thanks Q&A jzhang@cs.ccny.cuny.edu http://www-cs.ccny.cuny.edu/~jzhang/