Many slides taken from George Kollios, Boston University

Slides:



Advertisements
Similar presentations
1 Spatial Join. 2 Papers to Present “Efficient Processing of Spatial Joins using R-trees”, T. Brinkhoff, H-P Kriegel and B. Seeger, Proc. SIGMOD, 1993.
Advertisements

Spatial Indexing SAMs. Spatial Indexing Point Access Methods can index only points. What about regions? Z-ordering and quadtrees Use the transformation.
Spatial and Temporal Data Mining V. Megalooikonomou Spatial Access Methods (SAMs) II (some slides are based on notes by C. Faloutsos)
Multimedia Database Systems
Spatial Join Queries. Spatial Queries Given a collection of geometric objects (points, lines, polygons,...) organize them on disk, to answer point queries.
I/O-Algorithms Lars Arge Fall 2014 September 25, 2014.
Nearest Neighbor Queries using R-trees Based on notes from G. Kollios.
CMU SCS : Multimedia Databases and Data Mining Lecture #6: Spatial Access Methods Part III: R-trees C. Faloutsos.
2-dimensional indexing structure
Spatial Indexing SAMs. Spatial Indexing Point Access Methods can index only points. What about regions? Z-ordering and quadtrees Use the transformation.
Spatial Access Methods Chapter 26 of book Read only 26.1, 26.2, 26.6 Dr Eamonn Keogh Computer Science & Engineering Department University of California.
Spatial Indexing for NN retrieval
Spatial Indexing SAMs. Spatial Access Methods PAMs Grid File kd-tree based (LSD-, hB- trees) Z-ordering + B+-tree R-tree Variations: R*-tree, Hilbert.
Accessing Spatial Data
Spatial Indexing SAMs.
Spatial Queries Nearest Neighbor and Join Queries.
I/O-Algorithms Lars Arge University of Aarhus March 1, 2005.
Multi-dimensional Indexes
I/O-Algorithms Lars Arge Spring 2009 March 3, 2009.
Spatial Indexing I Point Access Methods. Spatial Indexing Point Access Methods (PAMs) vs Spatial Access Methods (SAMs) PAM: index only point data Hierarchical.
1 R-Trees for Spatial Indexing Yanlei Diao UMass Amherst Feb 27, 2007 Some Slide Content Courtesy of J.M. Hellerstein.
Spatial Indexing I Point Access Methods.
Spatial Queries Nearest Neighbor Queries.
I/O-Algorithms Lars Arge Aarhus University March 6, 2007.
©Silberschatz, Korth and Sudarshan12.1Database System Concepts Chapter 12: Part B Part A:  Index Definition in SQL  Ordered Indices  Index Sequential.
Spatial Queries. R-tree: variations What about static datasets? (no ins/del) Hilbert What about other bounding shapes?
R-Trees 2-dimensional indexing structure. R-trees 2-dimensional version of the B-tree: B-tree of maximum degree 8; degree between 3 and 8 Internal nodes.
Spatial Indexing SAMs. Spatial Access Methods PAMs Grid File kd-tree based (LSD-, hB- trees) Z-ordering + B+-tree R-tree Variations: R*-tree, Hilbert.
Spatial Indexing I Point Access Methods. Spatial Indexing Point Access Methods (PAMs) vs Spatial Access Methods (SAMs) PAM: index only point data Hierarchical.
Advanced Data Structures NTUA 2007 R-trees and Grid File.
Spatial Indexing I Point Access Methods. Spatial Indexing Point Access Methods (PAMs) vs Spatial Access Methods (SAMs) PAM: index only point data Hierarchical.
R-TREES: A Dynamic Index Structure for Spatial Searching by A. Guttman, SIGMOD Shahram Ghandeharizadeh Computer Science Department University of.
CMU SCS : Multimedia Databases and Data Mining Lecture #6: Spatial Access Methods Part III: R-trees C. Faloutsos.
R-Trees: A Dynamic Index Structure for Spatial Data Antonin Guttman.
Spatial Data Management Chapter 28. Types of Spatial Data Point Data –Points in a multidimensional space E.g., Raster data such as satellite imagery,
Database management Systems, 3ed, R. Ramakrishnan and J. Gehrke1 Spatial Data Management Chapter 28.
The X-Tree An Index Structure for High Dimensional Data Stefan Berchtold, Daniel A Keim, Hans Peter Kriegel Institute of Computer Science Munich, Germany.
Advanced Data Structures NTUA 2007 R-trees and Grid File.
Antonin Guttman In Proceedings of the 1984 ACM SIGMOD international conference on Management of data (SIGMOD '84). ACM, New York, NY, USA.
Nearest Neighbor Queries Chris Buzzerd, Dave Boerner, and Kevin Stewart.
Bin Yao (Slides made available by Feifei Li) R-tree: Indexing Structure for Data in Multi- dimensional Space.
Lecture 3: External Memory Indexing Structures (Contd) CS6931 Database Seminar.
Spatial Indexing Techniques Introduction to Spatial Computing CSE 5ISC Some slides adapted from Spatial Databases: A Tour by Shashi Shekhar Prentice Hall.
R-Trees: A Dynamic Index Structure For Spatial Searching Antonin Guttman.
1 CSIS 7101: CSIS 7101: Spatial Data (Part 1) The R*-tree : An Efficient and Robust Access Method for Points and Rectangles Rollo Chan Chu Chung Man Mak.
Spatial Queries Nearest Neighbor and Join Queries Most slides are based on slides provided By Prof. Christos Faloutsos (CMU) and Prof. Dimitris Papadias.
1 Spatial Query Processing using the R-tree Donghui Zhang CCIS, Northeastern University Feb 8, 2005.
1 R-Trees Guttman. 2 Introduction Range queries in multiple dimensions: Computer Aided Design (CAD) Geo-data applications Support special data objects.
Spatial Data Management
Strategies for Spatial Joins
Spatial Queries Nearest Neighbor and Join Queries.
Multiway Search Trees Data may not fit into main memory
Tree-Structured Indexes: Introduction
Chapter 25: Advanced Data Types and New Applications
Spatial Indexing.
Spatial Indexing I Point Access Methods.
Query Processing in Databases Dr. M. Gavrilova
KD Tree A binary search tree where every node is a
Nearest Neighbor Queries using R-trees
Database Applications (15-415) DBMS Internals- Part III Lecture 15, March 11, 2018 Mohammad Hammoud.
R-tree: Indexing Structure for Data in Multi-dimensional Space
Spatio-Temporal Databases
15-826: Multimedia Databases and Data Mining
Hash-Based Indexes Chapter 10
B+Trees The slides for this text are organized into chapters. This lecture covers Chapter 9. Chapter 1: Introduction to Database Systems Chapter 2: The.
Spatial Indexing I R-trees
File Processing : Multi-dimensional Index
Multidimensional Search Structures
Donghui Zhang, Tian Xia Northeastern University
C. Faloutsos Spatial Access Methods - R-trees
Presentation transcript:

Many slides taken from George Kollios, Boston University Spatial Indexing Many slides taken from George Kollios, Boston University

Spatial Indexing Point Access Methods can index only points. What about regions? Use the transformation technique Z-ordering and quadtrees New methods: Spatial Access Methods SAMs R-tree and variations

Problem Given a collection of geometric objects (points, lines, polygons, ...) organize them on disk, to answer spatial queries (range, nn, etc)

R-tree z-ordering: cuts regions to pieces -> dup. elim. how could we avoid that? Idea: try to extend/merge B-trees and k-d trees R-tree: a generalization of the B+-tree for multidimensional spaces

Kd-Btrees [Robinson, 81]: if f is the fanout, split point-set in f parts; and so on, recursively

Kd-Btrees But: insertions/deletions are tricky (splits may propagate downwards and upwards) no guarantee on space utilization

R-trees [Guttman 84] Main idea: allow parents to overlap! => guaranteed 50% utilization => easier insertion/split algorithms. (only deal with Minimum Bounding Rectangles - MBRs)

R-trees A multi-way external memory tree Index nodes and data (leaf) nodes All leaf nodes appear on the same level Every node contains between m and M entries The root node has at least 2 entries (children)

Example eg., w/ fanout 4: group nearby rectangles to parent MBRs; each group -> disk page I C A G F H B J E D

Example F=4 P1 P3 I C A G F H B J A B C H I J E P4 P2 D D E F G

Example F=4 P1 P3 I C A G F H B J E P4 P2 D P1 P2 P3 P4 A B C H I J D

R-trees - format of nodes {(MBR; obj_ptr)} for leaf nodes P1 P2 P3 P4 x-low; x-high y-low; y-high ... obj ptr A B C ...

R-trees - format of nodes {(MBR; node_ptr)} for non-leaf nodes x-low; x-high y-low; y-high ... node ptr P1 P2 P3 P4 ... A B C

R-trees:Search P1 P3 I C A G F H B J E P4 P2 D P1 P2 P3 P4 A B C H I J

R-trees:Search P1 P3 I C A G F H B J E P4 P2 D P1 P2 P3 P4 A B C H I J

R-trees:Search Main points: every parent node completely covers its ‘children’ a child MBR may be covered by more than one parent - it is stored under ONLY ONE of them. (ie., no need for dup. elim.) a point query may follow multiple branches. everything works for any(?) dimensionality

R-trees:Insertion Insert X P1 P3 I C A G F H B X J E P4 P2 D P1 P2 P3

R-trees:Insertion Insert Y P1 P3 I C A G F H B J Y E P4 P2 D P1 P2 P3

R-trees:Insertion Extend the parent MBR P1 P3 I C A G F H B J Y E P4

R-trees:Insertion How to find the next node to insert the new object? Using ChooseLeaf: Find the entry that needs the least enlargement to include Y. Resolve ties using the area (smallest) Other methods (later)

R-trees:Insertion P1 K P3 I C A G W F H B J E P4 P2 D If node is full then Split : ex. Insert w P1 K P3 I P1 P2 P3 P4 C A G W F H B J A B C K H I J E P4 P2 D D E F G

R-trees:Split (A1: plane sweep, until 50% of rectangles) P1 K Split node P1: partition the MBRs into two groups. (A1: plane sweep, until 50% of rectangles) A2: ‘linear’ split A3: quadratic split A4: exponential split: 2M-1 choices P1 K C A W B

R-trees:Split seed2 R seed1 pick two rectangles as ‘seeds’; assign each rectangle ‘R’ to the ‘closest’ ‘seed’ seed2 R seed1

R-trees:Split seed2 R seed1 pick two rectangles as ‘seeds’; assign each rectangle ‘R’ to the ‘closest’ ‘seed’: ‘closest’: the smallest increase in area seed2 R seed1

R-trees:Split How to pick Seeds: Linear:Find the highest and lowest side in each dimension, normalize the separations, choose the pair with the greatest normalized separation Quadratic: For each pair E1 and E2, calculate the rectangle J=MBR(E1, E2) and d= J-E1-E2. Choose the pair with the largest d

R-trees:Insertion Use the ChooseLeaf to find the leaf node to insert an entry E If leaf node is full, then Split, otherwise insert there Propagate the split upwards, if necessary Adjust parent nodes

R-Trees:Deletion Other method (later) Find the leaf node that contains the entry E Remove E from this node If underflow: Eliminate the node by removing the node entries and the parent entry Reinsert the orphaned (other entries) into the tree using Insert Other method (later)

R-trees: Variations R+-tree: DO not allow overlapping, so split the objects (similar to z-values) R*-tree: change the insertion, deletion algorithms (minimize not only area but also perimeter, forced re-insertion ) Hilbert R-tree: use the Hilbert values to insert objects into the tree

R-trees - variations what about static datasets (no ins/del/upd)? Q: Best way to pack points?

R-trees - variations what about static datasets (no ins/del/upd)? Q: Best way to pack points? A1: plane-sweep great for queries on ‘x’; terrible for ‘y’

R-trees - variations what about static datasets (no ins/del/upd)? Q: Best way to pack points? A1: plane-sweep great for queries on ‘x’; bad for ‘y’

R-trees - variations what about static datasets (no ins/del/upd)? Q: Best way to pack points? A1: plane-sweep great for queries on ‘x’; terrible for ‘y’ Q: how to improve?

R-trees - variations A: plane-sweep on HILBERT curve!

R-trees - variations A: plane-sweep on HILBERT curve! In fact, it can be made dynamic (how?), as well as to handle regions (how?)

R-trees - variations Dynamic (‘Hilbert R-tree): each point has an ‘h’-value (hilbert value) insertions: like a B-tree on the h-value but also store MBR, for searches

R-trees - variations what about other bounding shapes? (and why?) A1: arbitrary-orientation lines (cell-tree, [Guenther] A2: P-trees (polygon trees) (MB polygon: 0, 90, 45, 135 degree lines)

R-trees - variations A3: L-shapes; holes (hB-tree) A4: TV-trees [Lin+, VLDB-Journal 1994] A5: SR-trees [Katayama+, SIGMOD97] (used in Informedia)

R-trees - conclusions Popular method; like multi-d B-trees guaranteed utilization good search times (for low-dim. at least) Informix ships DataBlade with R-trees

Spatial Queries

Spatial Queries Given a collection of geometric objects (points, lines, polygons, ...) organize them on disk, to answer point queries range queries k-nn queries spatial joins (‘all pairs’ queries)

Spatial Queries Given a collection of geometric objects (points, lines, polygons, ...) organize them on disk, to answer point queries range queries k-nn queries spatial joins (‘all pairs’ queries)

Spatial Queries Given a collection of geometric objects (points, lines, polygons, ...) organize them on disk, to answer point queries range queries k-nn queries spatial joins (‘all pairs’ queries)

Spatial Queries Given a collection of geometric objects (points, lines, polygons, ...) organize them on disk, to answer point queries range queries k-nn queries spatial joins (‘all pairs’ queries)

Spatial Queries Given a collection of geometric objects (points, lines, polygons, ...) organize them on disk, to answer point queries range queries k-nn queries spatial joins (‘all pairs’ queries)

R-trees - Range search pseudocode: check the root for each branch, if its MBR intersects the query rectangle apply range-search (or print out, if this is a leaf)

R-trees - NN search A B C D E F G H I J P1 P2 P3 P4 q

R-trees - NN search Q: How? (find near neighbor; refine...) A B C D E J P1 P2 P3 P4 q

R-trees - NN search A1: depth-first search; then, range query P1 P3 I B J E P4 q P2 D

R-trees - NN search A1: depth-first search; then, range query P1 P3 I B J E P4 q P2 D

R-trees - NN search A1: depth-first search; then, range query P1 P3 I B J E P4 q P2 D

R-trees - NN search A2: [Roussopoulos+, sigmod95]: priority queue, with promising MBRs, and their best and worst-case distance main idea: Every face of any MBR contains at least one point of an actual spatial object!

R-trees - NN search consider only P2 and P4, for illustration A B C D G H I J P1 P2 P3 P4 q

R-trees - NN search best of P4 => P4 is useless for 1-nn worst of P2 H J E P4 q P2 D

R-trees - NN search what is really the worst of, say, P2? worst of P2 q P2 D

R-trees - NN search what is really the worst of, say, P2? A: the smallest of the two red segments! q P2

Nearest-neighbor searching Branch and bound strategy Compute MINDIST and MINMAXDIST [RKV95] MINDIST(p,R) is the minimum distance between p and R with corner points l and u the closest point in R is at least this distance away R u Sqrt sum (pi-ri)2 where p ri = li if pi < li = ui if pi > ui = pi otherwise p MINDIST = 0 l p

Nearest-neighbor searching MINMAXDIST(p,R) is the minimum of the maximum distance to each pair of faces of R MaxDistanceToFace(p,R,k) = distance between p and Maxk= (M1,M2,..,Mk-1mk,MK+1,..,Mn) mi = closer of the two boundary points along i axis Mi = farther of the two boundary points along i axis Max2 u p l Max1

Nearest-neighbor searching MINMAXDIST(p,R) is the minimum of the maximum distance to each pair of faces of R MaxDistanceToFace(p,R,k) = distance between p and Maxk= (M1,M2,..,Mk-1mk,MK+1,..,Mn) mi = closer of the two boundary points along i axis Mi = farther of the two boundary points along i axis Max1 u l p Max2

Pruning ESTIMATE := smallest MINMAXDIST(p,R) Prune an MBR R’ for which MINDIST(p,R’) is greater than ESTIMATE. Generalize to k-nearest neighbor searching Maintain kth largest MINMAXDIST Prune an MBR if MINDIST to it is larger than the current estimate of kth MINMAXDIST Can use objects to refine estimate

Order of searching Depth first order Inspect children in MINDIST order For each node in the tree keep a list of nodes to be visited Prune some of these nodes in the list Continue until the lists are empty

Another NN search Global order [HS99] Maintain distance to all entries in a common list Order the list by MINDIST Repeat Inspect the next MBR in the list Add the children to the list and reorder Until all remaining MBRs can be pruned

Spatial Join Find all parks in a city Find all trails that go through a forest Basic operation find all pairs of objects that overlap Single-scan queries nearest neighbor queries, range queries Multiple-scan queries spatial join

Algorithms No existing index structures Transform data into 1-d space [O89] z-transform; sensitive to size of pixel Partition-based spatial-merge join [PW96] partition into tiles that can fit into memory plane sweep algorithm on tiles Spatial hash joins [LR96, KS97] Sort data [BBKK01] With index structures [BKS93, HJR97] k-d trees and grid files R-trees

R-tree based Join [BKS93]

Join1(R,S) Repeat Find a pair of intersecting entries E in R and F in S If R and S are leaf pages then add (E,F) to result-set Else Join1(E,F) Until all pairs are examined CPU and I/O bottleneck

Reducing CPU bottleneck S R

Join2(R,S,IntersectedVol) Repeat Find a pair of intersecting entries E in R and F in S that overlap with IntersectedVol If R and S are leaf pages then add (E,F) to result-set Else Join2(E,F,CommonEF) Until all pairs are examined 14+6 comparisons instead of 49 In general, number of comparisons equals size(R) + size(S) + relevant(R)*relevant(S) Reduce the product term

Using Plane Sweep S R s1 s2 r1 r2 r3 Consider the extents along x-axis Start with the first entry r1 sweep a vertical line

Using Plane Sweep S R s1 s2 r1 r2 r3 Check if (r1,s1) intersect along y-dimension Add (r1,s1) to result set

Using Plane Sweep S R s1 s2 r1 r2 r3 Check if (r1,s2) intersect along y-dimension Add (r1,s2) to result set

Using Plane Sweep S R s1 s2 r1 r2 r3 Reached the end of r1 Start with next entry r2

Using Plane Sweep S R s1 s2 r1 r2 r3 Reposition sweep line

Using Plane Sweep S R s1 s2 r1 r2 r3 Check if r2 and s1 intersect along y Do not add (r2,s1) to result

Using Plane Sweep S R s1 s2 r1 r2 r3 Reached the end of r2 Start with next entry s1

Using Plane Sweep S R s1 s2 r1 r2 r3 Total of 2(r1) + 1(r2) + 0 (s1)+ 1(s2)+ 0(r3) = 4 comparisons

Reducing I/O Read schedule r1, s1, s2, r3 Every subtree examined only once Consider a slightly different layout

Reducing I/O S R s1 r2 r1 s2 r3 Read schedule is r1, s2, r2, s1, s2, r3 Subtree s2 is examined twice

Pinning of nodes After examining a pair (E,F), compute the degree of intersection of each entry degree(E) is the number of intersections between E and unprocessed rectangles of the other dataset If the degrees are non-zero, pin the pages of the entry with maximum degree Perform spatial joins for this page Continue with plane sweep

Reducing I/O S R s1 r2 r1 s2 r3 After computing join(r1,s2), degree(r1) = 0 degree(s2) = 1 So, examine s2 next Read schedule = r1, s2, r3, r2, s1 Subtree s2 examined only once

References [SK98] Optimal multi-step k-nearest neighbor search, T. Seidl and H. Kriegel, SIGMOD 1998: 154--165. [BBKK01] Epsilon Grid Order: An Algorithm for the Similarity Join on Massive High-Dimensional Data, C. Bohm, B. Braunmüller, F. Krebs and H.-P. Kriegel, SIGMOD 2001. [RKV95] Roussopoulos N., Kelley S., Vincent F. Nearest Neighbor Queries. Proceedings of the ACM-SIGMOD International Conference on Management of Data, pages 71-79, 1995.

References [HS99] G. R. Hjaltason and H. Samet, Distance browsing in spatial databases, ACM Transactions on Database Systems 24, 2 (June 1999), 265-318 [O89] Jack A. Orenstein: Redundancy in Spatial Databases. SIGMOD Conference 1989: 294-305 [PW96] Jignesh M. Patel, David J. DeWitt: Partition Based Spatial-Merge Join. SIGMOD Conference 1996: 259-270 [LR96] Ming-Ling Lo, Chinya V. Ravishankar: Spatial Hash-Joins. SIGMOD Conference 1996: 247-258

References [KS97] Nick Koudas, Kenneth C. Sevcik: Size Separation Spatial Join. SIGMOD Conference 1997: 324-335 [HJR97] Yun-Wu Huang, Ning Jing, Elke A. Rundensteiner: Spatial Joins Using R-trees: Breadth-First Traversal with Global Optimizations. VLDB 1997, 396-405 [BKS93] Thomas Brinkhoff, Hans-Peter Kriegel, Bernhard Seeger: Efficient Processing of Spatial Joins Using R-Trees. SIGMOD Conference 1993: 237-246