1. Exploiting Type and Space in a Main Memory Query Engine Thomas Schwarz Matthias Grossmann, Daniela Nicklas, Bernhard Mitschang Universität Stuttgart, Institute of Parallel and Distributed Systems

2. University of Stuttgart Center of Excellence 627 2 Outline Motivation and Scenarios Index Structures Related Work Experiments Conclusion

3. University of Stuttgart Center of Excellence 627 3 City-Guide Szenario Query: How do I get to the closest hotel? Hotel Youth-HostelMuseum

4. University of Stuttgart Center of Excellence 627 4 Typical Data and Type Hierarchy 1 2 3 4 6 7 Typical data Root0 Building1 Museum 3 Res- taurant 4 Road5 Local Road 6 Main Road 7 Hotel 2 Typical type hierarchy NameID Type Youth Hostel 8

5. University of Stuttgart Center of Excellence 627 5 Type Hierarchy of TIGER/Line Data Sets D51130 Airport or airfield D52131 Train station D53132 Bus terminal D97 Landmark A115 Primary..., unseparated A159 Primary..., separated A1913 Prim.., bridge A14 Primary Highway With Limited Access A434 Local, Neighborhood, and Rural Road D5128 Transpor- tation Terminal D4122 Educational or religious Institution B166 Railroad Main Line B1268..., in tunnel A1 Road B63 Railroad H213 Hydrography 0 the root type CFCCType ID description 258 types

6. University of Stuttgart Center of Excellence 627 6 Typical Queries Typical queries ask for Gas stations next to the planned route Nearest base stations for wireless internet Sights / landmarks / buildings in a given area All roads / only major roads in a given area Disjunctive queries Restrict type of queried objects Restrict location of queried objects Exploit these characteristics for speedup Leverage a dedicated index structure Combine both primary access paths

7. University of Stuttgart Center of Excellence 627 7 System Architecture Data Provider Mobile Device Application Discovery Service Integration Middleware Data Provider Mobile Device Application main memory query engine

8. University of Stuttgart Center of Excellence 627 8 Selectivity Factor Universe Query area Very selective, -but- Low selectivity factor: 20% Universe Query area Not very selective, -but- High selectivity factor: 70%

9. University of Stuttgart Center of Excellence 627 9 Usage Scenarios Data Provider All hotels of a brand in the entire country Few updates Discovery Service Metadata on diverse data providers Few updates Mobile Device Data for single application, around the user Many updates Integration Middleware Diverse data relevant in the range of the base station Many updates Spatial selectivity factor Type selectivity factor 1%10%100% 1% 10% 100%

10. University of Stuttgart Center of Excellence 627 10 Summary of the Requirements Simple query capabilitites suffice Combine Type and Space Cope with different workloads Fast response times

11. University of Stuttgart Center of Excellence 627 11 Outline Motivation and Scenarios Index Structures Related Work Experiments Conclusion

12. University of Stuttgart Center of Excellence 627 12 Separate Indexes Array (Main road) 7 (Hotel) 2 (Local road) 6 (Restaurant) 4 (Root) 0 (Museum) 3 (Road) 5 (Building) 1 Spatial index (Quadtree) chooses Separate Lists Cost-based optimizer Candidates Type predicate Spatial predicate Final Result

13. University of Stuttgart Center of Excellence 627 13 Real 3D Index 2 3 4 1..4 Building + all subtypes type dimension Query

14. University of Stuttgart Center of Excellence 627 14 Traversing the Index Spatial dimension Type dimension Query

15. University of Stuttgart Center of Excellence 627 15 Traversing the Index Spatial dimension Type dimension Query

16. University of Stuttgart Center of Excellence 627 16 Traversing the Index Spatial dimension Type dimension Query

17. University of Stuttgart Center of Excellence 627 17 Traversing the Index Spatial dimension Type dimension Query

18. University of Stuttgart Center of Excellence 627 18 Traversing the Index Spatial dimension Type dimension Query

19. University of Stuttgart Center of Excellence 627 19 Traversing the Index Spatial dimension Type dimension Query

20. University of Stuttgart Center of Excellence 627 20 Traversing the Index Spatial dimension Type dimension Query

21. University of Stuttgart Center of Excellence 627 21 Traversing the Index Spatial dimension Type dimension Query

22. University of Stuttgart Center of Excellence 627 22 Traversing the Index Spatial dimension Type dimension Query

23. University of Stuttgart Center of Excellence 627 23 Type Hierarchy Linearization Treat type information like a spatial dimension Root0 Building1 Museum 3 Res- taurant 4 Road5 Local Road 6 Main Road 7 Hotel 2

24. University of Stuttgart Center of Excellence 627 24 Type Hierarchy Linearization Treat type information like a spatial dimension Root0 Building1 Museum 3 Res- taurant 4 Road5 Local Road 6 Main Road 7 Hotel 2 Type dimension Building + all subtypes

25. University of Stuttgart Center of Excellence 627 25 Effects of the Spacing in the Type Dimension type dimension spatial dimension 02468 0 4 2 6 8 Objects are primarily grouped by their type Objects are primarily grouped by their position inner node of index tree object spatial dimension type dimension 02468 0 6 9 3 wide spacing between mapped values narrow spacing between mapped values Affects clustering of objects Determine best type mapping range

26. University of Stuttgart Center of Excellence 627 26 Type Mapping Variant: Equal Spread (ES) 01000100200300400500600700800900 0123456789 Type ID mapped value type mapping range range containing all subtypes Same gap between all mapped values 976432 15 8 0 The simplest variant

27. University of Stuttgart Center of Excellence 627 27 Type Mapping Variant: Type Hierarchy (TH) 01000 0 250 1 313 2 375 3 438 4 500 5 583 6 666 7 750 8 875 9 Type ID mapped value range containing all subtypes type mapping range Same gap between a type and its direct subtypes 976432 15 8 0 Cluster objects with same supertype

28. University of Stuttgart Center of Excellence 627 28 Type Mapping Variant: Object Distribution (OD) 01000 36 71250 304 429571643750839 0 1 23456789 Type ID mapped value range containing all subtypes Size of gap corresponds to the number of instances of a type 2 2 103 7 2 4 7 3 3 6 4 7 6 9 9 1 2 5 8 8 5 0 2 Cluster infrequent objects by location, cluster frequent objects by type type mapping range Requires additional histogram information

29. University of Stuttgart Center of Excellence 627 29 Related Work Spatial Indexes We use them, but don‘t build one Object-oriented Databases Use only point access methods Object-relational Databases Separate table for each type Query many tables for all subtypes Single global table Use point access methods

30. University of Stuttgart Center of Excellence 627 30 Outline Motivation and Scenarios Index Structures Related Work Experiments Conclusion

31. University of Stuttgart Center of Excellence 627 31 Experimental Setup Data sets from 9 counties in California (TIGER/Line 2003) Universe Width: 15 to 100 km Height: 26 to 115 km 12k to 203k objects 258 types

32. University of Stuttgart Center of Excellence 627 32 Total Average Query and Update Response Time Weighted query response time Usage Scenarios -> integration borders Type selectivity Spatial selectivity

33. University of Stuttgart Center of Excellence 627 33 Comparing the Type Mapping Ranges 100% 110% 120% 130% 140% 150% 160% ESTHODESTHODESTHODESTHOD Data Provider Discovery Service Integration Middleware Mobile Device Relative Response Time A, = 15km B, = 150km C, = 1500km D, = 15000km E, = 60000km type mapping range Almost best type mapping range is sufficient ^ ^ ^ ^ ^

34. University of Stuttgart Center of Excellence 627 34 Comparing the Approaches 100% 110% 120% 130% 140% Data Provider Discovery Service Integration Middleware Mobile Device 150% 200% 250% 300% 350% 400% 450% 500% 550% 600% 650% 700% 481% 518% 690% 446% Relative Response Time SEP indexing approach R3D.1:1 R3D.ES R3D.TH R3D.OD Type mapping does matter! Object Density is the best variant More impact with low type selectivity

35. University of Stuttgart Center of Excellence 627 35 Resource Consumption 0 5 10 15 20 25 12k17k23k46k 54k 72k 151k175k203k Bytes Nano seconds SEPR3D.OD.BR3D.OD.CR3D.OD.DR3D.OD.E Indexing approach Data set size 12k17k23k46k54k72k 151k175k 203k 0 100 200 300 400 500 600 Insertion time per objectMemory per object Scales well with larger data sets Speed costs resources

36. University of Stuttgart Center of Excellence 627 36 Conclusion Location-conscious main memory query engine Exploits characteristics of typical queries Deployable to many components Real 3D Index Best performance Type mapping range: Larger than expected Type mapping variant: Object Density Separate Indexes Best resource consumption

37. University of Stuttgart Center of Excellence 627 37 Outlook Virtualize query processing Dynamically distribute query capabilites according to load Integrate other dimensions Valid time Measurement time