1. 1 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 1 Advanced databases – Temporal and spatial databases II: Spatial databases Bettina Berendt Katholieke Universiteit Leuven, Department of Computer Science http://www.cs.kuleuven.be/~berendt/teaching/2007w/adb/ Last update: 17 October 2007

2. 2 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 2 Agenda Spatial DBMS, Spatial databases, GIS Space: Conceptual modelling issues Spatial queries and query languages Storage and indexing Query processing and optimization Spatial network models

3. 3 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 3 Purpose of this overview This slide set is a broad overview of spatial databases / DBMS For more details on specific questions, see „background reading“ at the end of the slides Recall - Spatial indexes were treated in Henk Olivié, 2006. Gegevensbanken: 13: organisatie van meerdimensionale structuren n Point quadtrees n Kd-trees n R-trees

4. 4 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 4 What is a SDBMS ? A SDBMS is a software module that n can work with an underlying DBMS n supports spatial data models, spatial abstract data types (ADTs) and a query language from which these ADTs are callable n supports spatial indexing, efficient algorithms for processing spatial operations, and domain specific rules for query optimization Example: Oracle Spatial data cartridge, ESRI SDE n can work with Oracle 8i DBMS n Has spatial data types (e.g. polygon), operations (e.g. overlap) callable from SQL3 query language n Has spatial indices, e.g. R-trees

5. 5 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 5 SDBMS Example Consider a spatial dataset with: n County boundary (dashed white line) n Census block - name, area, population, boundary (dark line) n Water bodies (dark polygons) n Satellite Imagery (gray scale pixels) Storage in a SDBMS table: create table census_blocks ( name string, area float, population number, boundarypolygon );

6. 6 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 6 Modeling Spatial Data in Traditional DBMS A row in the table census_blocks Question: Is Polyline datatype supported in DBMS?

7. 7 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 7 Spatial Data Types and Traditional Databases Traditional relational DBMS n Support simple data types, e.g. number, strings, date n Modeling Spatial data types is tedious Example: Figure 1.4 shows modeling of polygon using numbers n Three new tables: polygon, edge, points l Note: Polygon is a polyline where last point and first point are same n A simple unit square represented as 16 rows across 3 tables n Simple spatial operators, e.g. area(), require joining tables n Tedious and computationally inefficient

8. 8 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 8 Spatial Data Types and Post-relational Databases Post-relational DBMS n Support user defined abstract data types n Spatial data types (e.g. polygon) can be added Choice of post-relational DBMS n Object oriented (OO) DBMS n Object relational (OR) DBMS A spatial database is a collection of spatial data types, operators, indices, processing strategies, etc. and can work with many post- relational DBMS as well as programming languages like Java, Visual Basic etc.

9. 9 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 9 How is a SDBMS different from a GIS ? GIS is a software to visualize and analyze spatial data using spatial analysis functions such as Search Thematic search, search by region, (re-)classification Location analysis Buffer, corridor, overlay Terrain analysis Slope/aspect, catchment, drainage network Flow analysis Connectivity, shortest path Distribution Change detection, proximity, nearest neighbor Spatial analysis/Statistics Pattern, centrality, autocorrelation, indices of similarity, topology: hole description Measurements Distance, perimeter, shape, adjacency, direction GIS uses SDBMS n to store, search, query, share large spatial data sets

10. 10 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 10 How is a SDBMS different from a GIS ? SDBMS focuses on n Efficient storage, querying, sharing of large spatial datasets n Provides simpler set based query operations n Example operations: search by region, overlay, nearest neighbor, distance, adjacency, perimeter etc. n Uses spatial indices and query optimization to speedup queries over large spatial datasets. SDBMS may be used by applications other than GIS n Astronomy, Genomics, Multimedia information systems,... Will one use a GIS or a SDBM to answer the following: n How many neighboring countries does USA have? n Which country has highest number of neighbors?

11. 11 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 11 Agenda Spatial DBMS, Spatial databases, GIS Space: Conceptual modelling issues Spatial queries and query languages Storage and indexing Query processing and optimization Spatial network models

12. 12 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 12 Models of Spatial Information Two common models n Field based n Object based Example: Forest stands n Fig. 2.1 n (a) forest stand map n (b) Object view has 3 polygons n (c ) Field view has a function We will concentrate on the Object model in the following.

13. 13 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 13 Classifying Spatial objects Spatial Object TypesExample ObjectDimension PointCity0 CurveRiver1 SurfaceCountry2 Spatial objets are spatial attributes of general objects Spatial objects are of many types Simple 0- dimensional (points), 1 dimensional (curves), 2 dimensional (surfaces) Example: see table below Collections Polygon collection (e.g. boundary of Japan or Hawaii), … …

14. 14 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 14 Spatial Object Types in OGIS Data Model Each rectangle shows a distinct spatial object type

15. 15 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 15 Classifying Operations on spatial objects in Object Model Set theory basedUnion, Intersection, Containment, TopologicalTouches, Disjoint, Overlap, etc. DirectionalEast,North-West, etc. Metric (aka Geometrical)Distance Classifying operations Set based: 2-dimensional spatial objects (e.g. polygons) are sets of points a set operation (e.g. intersection) of 2 polygons produce another polygon Topological operations: Boundary of USA touches boundary of Canada Directional: New York city is to east of Chicago Metric: Chicago is about 700 miles from New York city. Q? Identify classes of spatial operations not listed in this slide.

16. 16 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 16 Topological Relationships n invariant under elastic deformation (without tear, merge). n Two countries which touch each other in a planar paper map will continue to do so in spherical globe maps. Topology is the study of topological relationships Example queries with topological operations n What is the topological relationship between two objects A and B ? n Find all objects which have a given topological relationship to object A ?

17. 17 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 17 Topological Concepts Interior, boundary, exterior n Let A be an object in a “Universe” U. A U Green is A interior Red is boundary of A Blue –(Green + Red) is A exterior Question: Define Interior, boundary, exterior on curves and points.

18. 18 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 18 Nine-Intersection Model of Topological Relationships Many topological Relationship between A and B can be specified using 9 intersection model Examples on next slide Nine intersections intersections between interior, boundary, exterior of A, B A and B are spatial objects in a two dimensional plane. Can be arranged as a 3 by 3 matrix Matrix element take a value of 0 (false) or 1 (true). Q? Determine the number of many distinct 3 by 3 boolean matrices.

19. 19 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 19 Specifying topological relations in the 9-Intersection Model 9-intersection matrices for exemplary topological operations (missing in this picture: TOUCH --The boundaries intersect but the interiors do not.) Note: The 9-intersection model is an „undirected“ version of the 13 Allen interval relations used for modelling time. Source: EGENHOFER, M.F. and FRANZOSA. Point Set Topological Spatial Relations. International Journal of Geographical Information Systems, vol 5, no 2, 161-174, 1991

20. 20 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 20 Using Object Model of Spatial Data Object model of spatial data n OGIS standard set of spatial data types and operations n Similar to the object model in computer software n Easily used with many computer software systems n Programming languages like Java, C++, Visual basic l Example use in a Java program is in section 2.1.6 n Post-relational databases, e.g. OODBMS, ORDBMS l Example usage in chapter 3 through 6

21. 21 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 21 Extending ER with Spatial Concepts Motivation ER Model is based on discrete sets with no implicit relationships Spatial data comes from a continuous set with implicit relationships Any pair of spatial entities has relationships like distance, direction, … Explicitly drawing all spatial relationship clutters ER diagram generates additional tables in relational schema Misses implicit constraints in spatial relationships (e.g. partition) Pictograms Label spatial entities along with their spatial data types Allows inference of spatial relationships and constraints Reduces clutter in ER diagram and relational schema Example: Compare the next two ER diagrams

22. 22 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 22 ER Diagram for “State-Park”: “non-spatial” version

23. 23 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 23 ER Diagram with Pictograms for “State-Park”

24. 24 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 24 Specifying Pictograms Grammar based approach Rewrite rule like English syntax diagrams Classes of pictograms Entity pictograms basic: point, line, polygon collection of basic... Relationship pictograms partition, network

25. 25 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 25 Entity Pictograms: Basic shapes, Collections

26. 26 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 26 Entity Pictograms: Derived and Alternate Shapes Derived shape example is city center point from boundary polygon Alternate shape example: A road is represented as a polygon for construction or as a line for navigation

27. 27 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 27 Corresponding changes to UML Class Diagrams

28. 28 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 28 ISO 19107:2003 Geographic information - Spatial schema n ISO 19107:2003 specifies conceptual schemas for describing the spatial characteristics of geographic features, and a set of spatial operations consistent with these schemas. n It treats vector geometry and topology up to three dimensions. n It defines standard spatial operations for use in access, query, management, processing, and data exchange of geographic information for spatial (geometric and topological) objects of up to three topological dimensions embedded in coordinate spaces of up to three axes. n https://committees.standards.org.au/COMMITTEES/IT- 004/PRIVATE/I0028/ISO%2019107%20.pdf https://committees.standards.org.au/COMMITTEES/IT- 004/PRIVATE/I0028/ISO%2019107%20.pdf n An ontology representing the Geographic Information - Spatial Schema (ISO 19107:2003) developed by Protégé (in OWL): http://loki.cae.drexel.edu/~wbs/ontology/iso-19107.htm http://loki.cae.drexel.edu/~wbs/ontology/iso-19107.htm

29. 29 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 29 Agenda Spatial DBMS, Spatial databases, GIS Space: Conceptual modelling issues Spatial queries and query languages Storage and indexing Query processing and optimization Spatial network models

30. 30 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 30 Problems of using SQL for Spatial Data Motivation n SQL has simple atomic data-types, like integer, dates and string n Not convenient for spatial data and queries l Spatial data (e.g. polygons) is complex l Spatial operation: topological, euclidean, directional, metric

31. 31 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 31 What kind of queries does one want to issue? Common Queries Point query: Find all rectangles containing a given point. Range query: Find all objects within a query rectangle. Nearest neighbor: Find the point closest to a query point. Intersection query: Find all the rectangles intersecting a query rectangle.

32. 32 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 32 Extending SQL for Spatial Data SQL 3 allows user defined data types and operations n Spatial data types and operations can be added to SQL3 Open Geodata Interchange Standard (OGIS) n Half a dozen spatial data types n Several spatial operations n Supported by major vendors, e.g. ESRI, Intergraph, Oracle, IBM,...

33. 33 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 33 OGIS Spatial Data Model Consists of base-class Geometry and four sub-classes: Point, Curve, Surface and GeometryCollection Operations fall into three categories: n Apply to all geometry types l SpatialReference, Envelope, Export,IsSimple, Boundary n Predicates for Topological relationships l Equal, Disjoint, Intersect, Touch, Cross, Within, Contains n Spatial Data Analysis l Distance,Buffer,Union, Intersection, ConvexHull, SymDiff

34. 34 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 34 Spatial Queries with SQL/OGIS SQL/OGIS - General Information Both standard are being adopted by many vendors The choice of spatial data types and operations is similar Syntax differs from vendor to vendor Readers may need to alter SQL/OGIS queries given in text to make them run on specific commercial products Using OGIS with SQL Spatial data types can be used in DML to type columns Spatial operations can be used in DML In the following, we will illustrate use of spatial data types with SQL, via a set of examples

35. 35 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 35 List of Spatial Query Examples Simple SQL SELECT_FROM_WHERE examples Spatial analysis operations Unary operator: Area Binary operator: Distance Boolean Topological spatial operations - WHERE clause Touch Cross Using spatial analysis and topological operations Buffer, overlap (not shown here) Aggregate SQL queries and nested queries can also be extended in similar ways (not shown here)

36. 36 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 36 Using spatial operation in SELECT clause Query: List the name, population, and area of each country listed in the Country table. SELECT C.Name,C.Pop, Area (C.Shape)AS "Area" FROM Country C Note: This query uses spatial operation, Area().Note the use of spatial operation in place of a column in SELECT clause.

37. 37 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 37 Using spatial operator Distance Query: List the GDP and the distance of a country’s capital city to the equator for all countries. SELECT Co.GDP, Distance (Point(0,Ci.Shape.y),Ci.Shape) AS "Distance" FROM Country Co,City Ci WHERE Co.Name = Ci.Country AND Ci.Capital =‘Y ’

38. 38 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 38 Using Spatial Operation in WHERE clause Query: Find the names of all countries which are neighbors of the United States (USA) in the Country table. SELECT C1.Name AS "Neighbors of USA" FROM Country C1,Country C2 WHERE Touch (C1.Shape,C2.Shape)=1 AND C2.Name =‘USA ’ Note: Spatial operator Touch() is used in WHERE clause to join Country table with itself. This query is an example of spatial self join operation.

39. 39 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 39 Spatial Query with multiple tables Query: For all the rivers listed in the River table, find the countries through which they pass. SELECT R.Name, C.Name FROM River R, Country C WHERE Cross (R.Shape,C.Shape)=1 Note: Spatial operation “Cross” is used to join River and Country tables. This query represents a spatial join operation.

40. 40 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 40 Agenda Spatial DBMS, Spatial databases, GIS Space: Conceptual modelling issues Spatial queries and query languages Storage and indexing Query processing and optimization Spatial network models

41. 41 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 41 Physical data model for SDBMS Is relational DBMS physical data model suitable for spatial data? n Relational DBMS has simple values like numbers n Sorting, search trees are efficient for numbers n These concepts are not natural for Spatial data (e.g. points in a plane) Reusing relational physical data model concepts n Space filling curves define a total order for points n This total order helps in using ordered files, search trees n But may lead to computational inefficiency! New spatial techniques n Spatial indices, e.g. grids, hierarchical collection of rectangles n Provide better computational performance

42. 42 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 42 Common assumptions for SDBMS physical model Spatial data n Dimensionality of space is low, e.g. 2 or 3 n Data types: OGIS data types n Approximations for extended objects (e.g. linestrings, polygons) l Minimum Orthogonal Bounding Rectangle (MOBR or MBR) l MBR(O) is the smallest axis-parallel rectangle enclosing an object O n Supports filter and refine processing of queries Spatial operations n OGIS operations, e.g. topological, spatial analysis n Many topological operations are approximated by “Overlap” n Common spatial queries - listed in next slide

43. 43 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 43 Common Spatial Queries and Operations Physical model provides simpler operations needed by spatial queries! Common Queries Point query: Find all rectangles containing a given point. Range query: Find all objects within a query rectangle. Nearest neighbor: Find the point closest to a query point. Intersection query: Find all the rectangles intersecting a query rectangle. Common operations across spatial queries find : retrieve records satisfying a condition on attribute(s) findnext : retrieve next record in a dataset with total order after the last one retrieved via previous find or findnext Nearest neighbor of a given object in a spatial dataset

44. 44 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 44 Recall: What is an index? Fig 4.10 Concept of an index auxiliary file to search a data file Example: Fig. 4.10 index records have key value address of relevant data sector see arrows in Fig. 4.10 Index records are ordered find, findnext, insert are fast Note assumption of total order on values of indexed attributes

45. 45 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 45 Attribute data types and Indices Index file structure depends on data type of indexed attribute Attributes with total order Example, numbers, points ordered by space filling curves B-tree is a popular index organization See Figure 1.12 (pp. 18) and section 1.6.4 Spatial objects (e.g. polygons) Spatial organization are more efficient Hundreds of organizations are proposed in literature Two main families are Grid Files and R-trees

46. 46 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 46 Ideas behind Grid Files Basic idea- Divide space into cells by a grid Example:  Example: latitude-longitude, ESRI Arc/SDE Store data in each cell in distinct disk sector Efficient for find, insert, nearest neighbor But may have wastage of disk storage space non-uniform data distribution over space Refinement of basic idea into Grid Files 1. Use non-uniform grids Linear scale store row and column boundaries 2. Allow sharing of disk sectors across grid cells See Figure 4.13 on next slide

47. 47 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 47 Grid Files Grid File component Linear scale - row/column boundaries Grid directory: cell --> disk sector address data sectors on disk Operation implementation Scales and grid directory in main memory Steps for find, nearest neighbor Search linear scales Identify selected grid directory cells Retrieve selected disk sectors Performance overview Efficient in terms of I/O costs Needs large main memory for grid directory

48. 48 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 48 R-Tree Family Basic Idea Use a hierarchical collection of rectangles to organize spatial data Generalizes B-tree to spatial data sets Classifying members of R-tree family Handling of large spatial objects Allow rectangles to overlap- R-tree Duplicate objects but keep interior node rectangles disjoint - R+tree Selection of rectangles for interior nodes greedy procedures - R-tree, R+tree procedure to minimize ocoverage, overlap - packed R-tree Other criteria exist Scope of our discussion Basics of R-tree and R+tree Focus on concepts not procedures!

49. 49 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 49 Spatial Objects with R-Tree Properties of R-trees Balanced Nodes are rectangle child’s rectangle within parent’s possible overlap among rectangles! Implementation of find operation Search root to identify relevant children Search selected children recursively Ex.: find record for rectangle 5 Root search identifies child x Search of x identifies children b and c Search of b does not find object 5 Search of c find object 5

50. 50 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 50 R+tree Properties of R+trees Balanced Interior nodes are rectangle child’s rectangle within parent’s disjoint rectangles Leaf nodes - MOBR of polygons or lines leaf’s rectangle overlaps with parent’s Data objects may be duplicated across leafs find operation - same as R-tree But only one child is followed down Ex.: find record for rectangle 5 Root search identifies child x Search of x identifies children b and c Search either b or c to find object 5

51. 51 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 51 Join Index Ideas Spatial join is a common operation. Expensive to compute using traditional indexes Spatial join index pre-computes and stores id-pairs of matched rows across tables Example see below Speeds up computation of spatial join

52. 52 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 52 Agenda Spatial DBMS, Spatial databases, GIS Space: Conceptual modelling issues Spatial queries and query languages Storage and indexing Query processing and optimization Spatial network models

53. 53 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 53 QPO Challenges in SDBMS Building Blocks for spatial queries n Rich set of spatial data types, operations n A consensus on “building blocks” is lacking n Current choices include spatial select, spatial join, nearest neighbor Choice of strategies n Limited choice for some building blocks, e.g. nearest neighbor Choosing best strategies n Cost models are more complex since l Spatial Queries are both CPU and I/O intensive l while traditional queries are I/O intensive n Cost models of spatial strategies are in not mature.

54. 54 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 54 Building Blocks for Spatial Queries Challenges in choosing building blocks n Rich set of data types - point, line string, polygon, … n Rich set of operators - topological, euclidean, set-based, … n Large collection of computation geometric algorithms l for different spatial operations on different spatial data types n Desire to limit complexity of SDBMS How to simplify choice of data types and operators? n Reusing a Geographic Information System (GIS) l which already implements spatial data types and operations l however may have difficulties processing large data set on disk n SDBMS reduces set of objects to be processed by a GIS n SDBMS is used as a filter n This is filter and refinement approach

55. 55 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 55 The Filter-Refine Paradigm Processing a spatial query Q Filter step : find a superset S of object in answer to Q Using approximate of spatial data type and operator Refinement step : find exact answer to Q reusing a GIS to process S Using exact spatial data type and operation

56. 56 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 56 Approximate Spatial Data types Approximating spatial data types n Minimum orthogonal bounding rectangle (MOBR or MBR) l approximates line string, polygon, … l See Examples below (Bblack rectangle are MBRs for red objects) n MBRs are used by spatial indexes, e.g. R-tree n Algorithms for spatial operations MBRs are simple

57. 57 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 57 Approximate Spatial Operations (1) Approximating spatial operations n SDBMS processes MBRs for refinement step n Overlap predicate used to approximate topological operations n Example: inside(A, B) replaced by l overlap(MBR(A), MBR(B)) in filter step l See picture below - Let A be outer polygon and B be the inner one l inside(A, B) is true only if overlap(MBR(A), MBR(B)) l However overlap is only a filter for inside predicate needing refinement later

58. 58 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 58 Filter Step Example Query: n List objects in front of a viewer V Equivalent overlap query n Direction region is a polygon n List objects overlapping with l polygon( front(V)) Approximate query n List objects overlapping with l MBR(polygon (front (V)))

59. 59 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 59 Approximate Spatial Operations (2) Exercise: Approximate following using overlap predicate n Cross(A, B), Touch(A, B), Disjoint(A, B) n See Table 3.9, pp. 66 for definition of these operations. Exercise: Given MBRs R and S, Provide conditions to test n Overlap(A, B) n Use coordinates of left-lower and upper-right corners of MBRs

60. 60 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 60 Choice of building blocks n Varies across software vendors and products n Representative building blocks are listed here List of building blocks n Point Query- Name a highlighted city on a digital map. l Return one spatial object out of a table n Range Query- List all countries crossed by of the river Amazon. l Returns several objects within a spatial region from a table n Spatial Join: List all pairs of overlapping rivers and countries. l Return pairs from 2 tables satisfying a spatial predicate n Nearest Neighbor: Find the city closest to Mount Everest. l Return one spatial object from a collection

61. 61 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 61 Strategies for Each Building Block Choice of strategies n Varies across software vendors and products n Representative strategies are listed here n Some strategies need special file-structures or indices Description of strategies n Main message: there are multiple strategies for each building block! n Focus on concepts rather than procedures n Readers interested in procedural details (e.g. algorithms) l Refer to papers in Bibliographic notes l Note: better algorithms appear in literature every year!

62. 62 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 62 Strategies for Point Queries Recall Point Query Example n Name a highlighted city on a digital map. n Return one spatial object out of a table List of strategies n Scan all B disk sectors of the data file n If records are ordered using space filling curve (say Z-order) l then use binary search on the Z-order of search point l to examine about log(B, base = 2) disk sectors n If an index is available on spatial location of data objects, l then use find() operation on the index l number of disk sector examined = depth of index (typically 4 to 5)

63. 63 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 63 Strategies for Range Queries Recall Range Query Example- n List all countries crossed by of the river Amazon. n Returns several objects within a spatial region from a table List of strategies n Scan all B disk sectors of the data file n If records are ordered using space filling curve (say Z-order) l then determine range of Z-order values satisfying range query l Use binary search to get lowest Z-order within query answer l Scan forward in the data file till the highest z-order satisfying query n If an index is available on spatial location of data objects, l then use range-query operation on the index

64. 64 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 64 Strategies for Spatial Joins Recall Spatial Join Example: n List all pairs of overlapping rivers and countries. n Return pairs from 2 tables satisfying a spatial predicate List of strategies n Nested loop: l Test all possible pairs for spatial predicate l All rivers are paired with all countries n Space Partitioning: l Test pairs of objects from common spatial regions only l Rivers in Africa are tested with countries in Africa only! n Tree Matching l Hierarchical pairing of object groups from each table n Other, e.g. spatial-join-index based, external plane-sweep, …

65. 65 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 65 Strategies for Nearest Neighbor Queries Recall Nearest Neighbor Example n Find the city closest to Mount Everest. n Return one spatial object from city data file C List of strategies n Two phase approach l Fetch C’s disk sector(s) containing the location of Mt. Everest l M = minimum distance( Mt. Everest, cities in fetched sectors) l Test all cities within distance M of Mt. Everest (Range Query) n Single phase approach l Recursive algorithm for R-tree l Eliminate candidates dominated by some other candidate

66. 66 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 66 Agenda Spatial DBMS, Spatial databases, GIS Space: Conceptual modelling issues Spatial queries and query languages Storage and indexing Query processing and optimization Spatial network models

67. 67 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 67 Motivation: Navigation Systems In-vehicle navigation systems n Services: l map destination given a street address l compute the shortest route to a destination l Help drivers follow selected route Recall that many maps and GIS were made for navigation n 16th century shipping, trade and treasure hunts n Maps were used to find destination and avoid hazards Navigation and transportation are important applications! Can OGIS spatial abstract data types support navigation?

68. 68 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 68 Navigation Systems and OGIS Spatial ADTs Can OGIS spatial abstract data types support navigation? OGIS spatial ADTs discussed in chapter 2 and 3 are inadequate for n computing route to a destination Rationale: Part 1 n Fact: Relation language (e.g. SQL2) can’t compute transitive closure! n Fact: Shortest path computation is an instance of transitive closure Rationale: Part 2 n OGIS spatial ADTs discussed in chapters 2 and 3 do not include l Graph data type, shortest_path operations See section 6.3.1, pp. 158 for details.

69. 69 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 69 Shortest path query Smallest travel time path between the two points shown in blue color. It follows a freeway (I-94) which is faster but not shorter in distance. Shortest distance path between the same two points shown in red colo, which is is hidden under blue on common edges. Spatial network query example

70. 70 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 70 Conceptual Data Models for spatial networks Conceptual Data Model for Spatial Networks A graph, G = (V,E) V = a finite set of vertices E = a set of edges E, between vertices in V Example: two graph models of a roadmap 1. Nodes = road-intersections, edges = road segment between intersections 2. Nodes = roads (e.g. Route 66), edge(A, B) = road A crosses road B Classifying graph models Do nodes represent spatial points? - spatial vs. abstract graphs Are vertex-pair in an edge order? - directed vs. undirected Example (continued) Model 1 is a spatial graph, Model 2 is an abstract graph Model 2 is undirected but can be directed or undirected

71. 71 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 71 Logical Data Model - Data types Common data types Vertex: attributes are label, isVisited, (location for spatial graphs) DirectedEdge : attributes are start node, end node, label Graph: attributes are setOfVertices, setOfDirectedEdges,... Path: attributes are sequenceOfVertices

72. 72 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 72 Logical Data Model - Operations Low level operations on a Graph G IsDirected - return true if and only if G is directed Add, AddEdge- adds a given vertex, edge to G Delete, DeleteEdge - removes specifies node (and related edges), edge from G Get, GetEdge- return label of given vertex, edge Get-a-successor, GetPredecessors - return start or end vertex of an edge GetSucessors- return end vertices of all edge starting at a given vertex Building blocks for queries shortest_path(vertex start, vertex end) shortest tour(vertex start, vertex end, setOfVertices stops) locate_nearest_server(vertex client, setOfVetices servers) allocate(setOfVetices servers)

73. 73 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 73 Physical Data Models Categories of record/file representations Main memory based Disk based Main memory representations of graphs Adjacency matrix M[A, B] = 1 if and only if edge(vertex A, vertex B) exists Adjacency list : maps vertex A to a list of successors of A Example: See Figure 6.2(a), (b) and ( c) on next slide Disk based normalized - tables, one for vertices, other for edges denormalized - one table for nodes with adjacency lists Example: See Figure 6.2(a), (d) and ( e) on next slide

74. 74 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 74 Query Languages For Graphs Recall Relation algebra (RA) based languages Can not compute transitive closure SQL3 provides support for transitive closure on graphs supports shortest paths SQL support for graph queries SQL2 - CONNECT clause in SELECT statement For directed acyclic graphs, e.g. hierarchies SQL 3 - WITH RECURSIVE statement Transitive closure on general graphs SQL 3 -user defined data types Can include shortest path operation!

75. 75 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 75 Query Processing for Spatial Networks Recall Query Processing and Optimization DBMS decomposes a query into building blocks Keeps a couple of strategy for each building block Selects most suitable one for a given situation Building blocks for graph transitive closure operations Connectivity(A, B) Is node B reachable from node A? Shortest path(A, B) Identify least cost path from node A to node B E.g., Dijkstra’s algorithm, Best First, hierarchical routing

76. 76 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 76 Next lecture Spatial DBMS, Spatial databases, GIS Space: Conceptual modelling issues Spatial queries and query languages Storage and indexing Query processing and optimization Spatial network models Combining heterogeneous databases, schemas and ontologies

77. 77 Berendt: Advanced databases, winter term 2007/08, http://www.cs.kuleuven.ac.be/~berendt/teaching/2007w/adb/ 77 References / background reading; acknowledgements Shekhar, S., & Chawla, S. (2003). Spatial Databases: A Tour. Upper Saddle River, NJ: Prentice Hall. Slides at http://www.spatial.cs.umn.edu/Book/slides/http://www.spatial.cs.umn.edu/Book/slides/ (All slides in this slide set except the agenda slides and slides 3 and 28 are taken from or based on these.) For more details on the R-trees family, see the original sources, in particular: n Antonin Guttman: R-Trees: A Dynamic Index Structure for Spatial Searching, Proc. 1984 ACM SIGMOD International Conference on Management of Data, pp. 47-57. http://www.sai.msu.su/~megera/postgres/gist/papers/gutman-rtree.pdf http://www.sai.msu.su/~megera/postgres/gist/papers/gutman-rtree.pdf n Norbert Beckmann, Hans- N. Beckmann, H.-P. Kriegel, R. Schneider, B. Seeger: The R*- Tree: An Efficient and Robust Access Method for Points and Rectangles. SIGMOD Conference 1990: 322-331 http://dbs.mathematik.uni- marburg.de/publications/myPapers/1990/BKSS90.pdfhttp://dbs.mathematik.uni- marburg.de/publications/myPapers/1990/BKSS90.pdf n T. Sellis, N. Roussopoulos, and C. Faloutsos. The R+-Tree: A dynamic index for multi- dimensional objects. In VLDB, 1987. http://citeseer.ist.psu.edu/sellis87rtree.htmlhttp://citeseer.ist.psu.edu/sellis87rtree.html n Lars Arge, Mark de Berg, Herman J.Haverkort, Ke Yi: The Priority R-Tree: A Practically Efficient and Worst-Case Optimal R-Tree, Proc. 2004 ACM SIGMOD international conference on Management of data, pp. 347-358. http://citeseer.ist.psu.edu/630317.htmlhttp://citeseer.ist.psu.edu/630317.html n Overview in: Yannis Manolopoulos, Alexandros Nanopoulos, Apostolos N. Papadopoulos, Yannis Theodoridis: R-Trees: Theory and Applications, Springer, 2005.Yannis ManolopoulosAlexandros NanopoulosApostolos N. PapadopoulosYannis Theodoridis