Efficient OLAP Operations for Spatial Data Using P-Trees Baoying Wang, Fei Pan, Dongmei Ren, Yue Cui, Qiang Ding William Perrizo North Dakota State University

OUTLINE Introduction Review Of Peano Trees (P-trees) OLAP Operations Using P-trees  Peano Data Cubes (PD-Cubes)  OLAP Operations Performance Analysis Conclusion

INTRODUCTION Efficient OLAP for spatial data warehouses is in great demand Spatial warehouses is growing with more and more spatial data, such as remotely sensed images, geographical information, digital sky survey data The data in a warehouse are conceptually modeled as data cubes (Gray et al, 1997)

INTRODUCTION (Cont.) OLAP queries are complex and time consuming Two major approaches to speed up OLAP  Using index structures  Operating on compressed data. Bitmap index are space inefficient for high cardinality attributes, and are only suitable for narrow domains.

Our Approach on OLAP A new data warehousing structure, PD- cube, is developed to facilitate OLAP operations and queries Fast logical operations of P-Trees are used to accomplish OLAP operations. Predicate P-trees are used to efficiently reduce data accesses by filtering out “big holes” consisting of consecutive 0’s

REVIEW OF PEANO TREES (Ptrees) The Ptree is a quadrant-based tree structure (assuming a 2-dimensional image; more generally, for n- dimensional data, an n-polytant tree) It is used to facilitate compression and very fast logical operations on bit sequential (bSQ) data (Perrizo, 2001)  Ptrees can be 1-dimensional, 2-dimensional, 3-dimensional… The most useful form of a Ptree is the predicate- Ptree: e.g., Pure1 Ptree (P1tree) and NonPure0 Ptree (NP0-tree)

bSQ File and a Pure1 tree (P1-tree) P1-tree: Tree node=1 iff that sub-quadrant is purely 1-bits

An Count Ptree (NOTE: usually counts are the ultimate goal, but Pure1 trees are easier to work with and produce the needed counts quite quickly) Peano or Z-ordering Pure (Pure-1/Pure-0) quadrant Root Count  Level  Fan-out  QID (Quadrant ID) ( 7, 1 ) ( 111, 001 )

BSQ File and a NP0-tree NP0-tree: Node=1 iff that sub-quadrant is not pure zero. (more general; -Ptree: node=1 iff sub-quad satisfies

Logical Operations of P-trees Operations are level by level Consecutive 0’s holes can be filtered out We only need to load quadrant with Qid 2 for ANDing NP0-tree1 and NP0-tree2.

OLAP OPERATIONS USING P-TREES 1.Peano Data Cube (PD-cube) 2.OLAP Operations 1)Slice/Dice 2)Rollup 3.Performance Analysis

Peano Data Cube (PD-cube) The data cube is partitioned by bit position  Each bit-wised data cube is in Peano order Take advantage of the continuity and sparseness of spatial data An example: a 3-D data cube representing the crop yield with three dimensions: X- coordinate, Y-coordinate, and time T.

A Fact Table and the PD-cubes XYTYield (1111) (0100) (0001) (1100) (0010) (1100) (1111) (0010) (0000) (1111) (0010)

OLAP Query Examples “Find all galaxies brighter than magnitude 22.” “Find average crop yield in a field. ” “Find area of the region with the color red.” “Find total traffic flow during a given period.”

Slice/Dice Operations Typical select statements may have a number of predicates in their “where” clause. The predicates may include “=”, “ ”.  These predicates lead to two different query scenarios: equal queries (“=”) and range queries (“ ”).

Equal Select Slice Example Suppose we have a 3-D data cube representing crop yield with dimensions X, Y and T, where X = {0, 1}, Y = {0, 1} and T = {0, 1}. XYTYield

Ptrees for 3-D Cube Example P ij is a Ptree for the j th bit of the i th attribute.

Slice: “Get yield where Y = 1” First get Ptree masks, and then trim all Ptrees accordingly

Range Slice: Get yield where Y >1001 Data set {“Y > 1001”} consists of two subsets {“Y = 11**”} and {“Y = 101*”}, where * is 1 or 0. The query clause can be written as “where Y = 11** || Y = 101*”. The query is retrieved by Ptree mask PM gt = PM1 || PM2. PM1 = P21 & P22 PM2 = P21 & P’22 & P ** 101* P21 & P22 P21 & P’22 & P23

Other Properties of Range Queries Combination of an Equality Query and a Range Query  Divide {“T  ”} into 2 subsets {“T> ”}, {“T= ”} Complement of a Range Query  Data set {“T  ”} is the complement of {“T> ”}  With the result of query “Get yield where T> , we can easily retrieve query “Get yield where T  ” by making the complement, i.e. PM le = PM’ gt.

Rollup Operations PD-cube is stored in Peano order rather than in raster order. Therefore, the rollup of PD-cube is accomplished differently from the traditional data cube as a result of different storage models. According to the Peano storage of PD- cube, we develop the recursive rollup algorithm.

Rollup of “Yield” along Dimension T S2[ ] = {1, 1, 1, 0} S1 [ ] = {0, 1, 1, 1} S0 [ ] = {2, 1, 1, 1} S [ ] = {8, 7, 7, 7} S[i] = S2[i] x S1[i] x S0[i] x

PERFORMANCE ANALYSIS: Compare our algorithm with bitmap indexed data cube method The data is prepared in five sizes, 128x128, 256x256, 512x512, 1024x1024, and 2048x2048. When cube size > 1900KB, our method outperforms bitmap indexed data cube method. As the cube size increases, there is a drastic increase in response time for bitmap indexed data cube method.

CONCLUSION A general spatial data warehousing structure, PD- cube, is presented to facilitate OLAP operations. The fast logical operations of Ptrees are used to accomplish these operations. Predicate Ptrees are used to find the “big holes” of consecutive 0’s by performing logical operations. Experiments indicate OLAP operations using Ptrees is much faster than traditional data cube methods.

FUTURE WORK One future research direction is to extend our PD-cube into parallel data warehouse systems.  It appears to be particularly promising to partition large cubes horizontally or vertically (or both) into small cubes to improve the query performance through parallelism.

Thank you.