© 2000 FORWISS, 1 MISTRAL Processing Relational Queries Using a Multidimensional Access Method Volker Markl Rudolf Bayer FORWISS (Bayerisches Forschungszentrum für Wissensbasierte Systeme)

© 2000 FORWISS, 2 Staff Members MISTRAL Project Management Prof. Rudolf Bayer, Ph.D. (FORWISS Knowledge Bases Group Head) Dr. Volker Markl (MISTRAL Project Leader, Deputy Research Group Head) MISTRAL Research Assistants Dipl. Inform. Robert Fenk Dipl. Inform. Roland Pieringer Frank Ramsak, M.Sc. Dipl. Inform. Martin Zirkel MISTRAL Master Students and Interns Ralf Acker, Bulent Altan, Sonja Antunes, Michael Bauer, Sascha Catelin, Naoufel Boulila, Nils Frielinghaus, Sebastian Hick, Stefan Krause, Jörg Lanzinger, Christian Leiter, Yiwen Lue, Stephan Merkel, Nasim Nadjafi, Oliver Nickel, Daniel Ovadya, Markus Pfadenauer, Timka Piric, Sabine Rauschendorfer, Antonius Salim, Maximilian Schramm, Michael Streichsbier, Anton Tichatschek

© 2000 FORWISS, Range Queries Tetris Algorithm

© 2000 FORWISS, 4 Overview 1. Concept of the UB-Tree: Z-Regions 2. Insertion 3. Range Query Algorithm 4. Tetris Algorithm 5. Kernel Integration 6. Performance Overview

© 2000 FORWISS, 5 Relations and MD Space l Decision Support Relation (similar to TPC-D) – Fact(customer, product, time, Sales) Ô defines a three dimensional cube l Point Query – All sales for one customer for one specific product on a certain day l Partial Match Query – All sales for product X l Range Query – All sales for year 1999 for a specific product group for a specific customer group

© 2000 FORWISS, 6 Design Goals l clustering tuples on disk pages while preserving spatial proximity l efficient incremental organization l logarithmic worst-case guarantees for insertion, deletion and point queries l efficient handling of range queries l good average memory utilization

© 2000 FORWISS, 7 Z-Ordering (a)(b)

© 2000 FORWISS, 8 Z-regions/UB-Trees A Z-region [  :  ] is the space covered by an interval on the Z-curve and is defined by two Z-addresses  and . UB-Tree partitioning: [0 : 3],[4 : 20], [21 : 35], [36 : 47], [48 : 63] Z-region [4 : 20] 4 20 point data creating the UB-Tree on the left for a page capacity of 2 points

© 2000 FORWISS, 9 UB-Tree Insertion 1/2/3/4

© 2000 FORWISS, 10 UB-Tree Insertion 18/19

© 2000 FORWISS, 11 Multidimensional Range Query SELECT * FROM table WHERE (A 1 BETWEEN a 1 AND b 1 ) AND (A 2 BETWEEN a 2 AND b 2 ) AND..... (A n BETWEEN a n AND b n )

© 2000 FORWISS, 12 Theoretical Comparison of the Rangequery Performance ideal case s 1 *s 2 *P multidimensional index s 1  *s 2  *P multiple B-Trees, bitmap indexes s 1 *I 1 +s 2 *I 2 +s 1 *s 2 *T composite key clustering B-Tree s 1 *P

© 2000 FORWISS, 13 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 14 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 15 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 16 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 17 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 18 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 19 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 20 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 21 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 22 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 23 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 24 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 25 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 26 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 27 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 28 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 29 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 30 rangeQuery(Tuple ql, Tuple qh ) { Zaddress start = Z( ql ); Zaddress cur = start ; Zaddress end = Z( qh ); Page page = {}; while (1) { cur = getRegionSeparator( cur ); page = getPage( cur ); outputMatchingTuples( page, ql, qh ); if ( cur >= end ) break; cur = getNextZAddress( cur, start, end ); } B * -Tree linear Z-space UB-Tree

© 2000 FORWISS, 31 Range Queries and Data Distributions

© 2000 FORWISS, 32 Growing Databases 1000 tuples tuples

© 2000 FORWISS, 33 Summary UB-Trees ü 50% storage utilization, dynamic updates ü Efficient Z-address calculation (bit-interleaving) ü Logarithmic performance for the basic operations ü Efficient range query algorithm (bit-operations) ü Prototype UB/API above RDBMS (Oracle 8, Informix, DB2 UDB, TransBase, MS SQL 7.0) using ESQL/C ? Patent application

© 2000 FORWISS, 34 Standard Query Pattern SELECT * FROM table WHERE (A 1 BETWEEN a 1 AND b 1 ) AND (A 2 BETWEEN a 2 AND b 2 ) AND..... (A n BETWEEN a n AND b n ) ORDER BY A i, A j, A k,... (GROUP BY A i, A j, A k,...)

© 2000 FORWISS, 35 Z-Order/Tetris Order A 2 A A 2 A 1 0 T j (x) = x j ∘ Z(x 1,...,x j-1,x j+1,...,x d ) A 2 A 1

© 2000 FORWISS, 36 The Tetris Algorithm sort direction

© 2000 FORWISS, 37 Summary Tetris l Combines sort process and evaluation of multi- attribute restrictions in one processing step l I/O-time linear w.r. to result set size l temporary storage sublinear w.r. to result set size l Sorting no longer a “blocking operation” ? Patent application

© 2000 FORWISS, 38 Integration Issues l Starting point with TransBase: – clustering B*-Tree – appropriate data type for Z-values: variable bit strings l Modifications to B*-Tree in TransBase: – support for computed keys: »Z-values are only stored in the index, not together with the tuples »tuples are stored in Z-order – generalization of splitting algorithm: »computed page separators for improved space partitioning

© 2000 FORWISS, 39 l Minor extensions: l Major extensions: l New modules: l NO changes for: – DML – Multi-user support, i.e., locking, logging facilities  handled by underlying B*- Tree

© 2000 FORWISS, 40 Summary Integration l Integration of the UB-Tree has been achieved within one year l TransBase HyperCube is shipping since Systems 1999 and was awarded the 2001 IT-Prize by EUROCASE and the European Commission l UB-Trees speed up relational DBMS for multidimensional applications like Geo-DB and data warehouse up to two orders of magnitude l Speedup is even more dramatic for CD-ROM databases (archives)

© 2000 FORWISS, 41 Application Fields of the UB-Tree l Data Warehouses – Measurements with SAP BW Data »UB-Tree/API for Oracle »UB-Tree on top of Oracle outperforms conventional B-Tree and Bitmap indexes in Oracle! – Measurements with the GfK Data Warehouse »UB-Tree in TransBase HyperCube »significant performance increases (Factor of 10) l Geographic Databases l „Multidimensional Problems“ – Archiving Systems, Lifecycle-Management, Data Mining, OLAP, OLTP, etc.

© 2000 FORWISS, 42 The UB-Tree

© 2000 FORWISS, 43 Further Information Tetris HI MISTRAL UB-Tree Te mp tris