ISO/IEC JTC1 SC32 1SQL/OLAP Sang-Won Lee Let’s e-Wha! URL: Jul. 12th,

ISO/IEC JTC1 SC32 2SQL/OLAP Contents  Introduction to OLAP and SQL Issues  Current OLAP Solutions  SQL/OLAP  Future OLAP Trends

ISO/IEC JTC1 SC32 3SQL/OLAP OLAP  On-Line Analytical Processing – E.F. Codd coined the term “OLAP”([1]) – Multi-dimensional data model – vs. On-Line Transaction Processing – vs. Data warehouse

ISO/IEC JTC1 SC32 4SQL/OLAP Data Warehouse Architecture

ISO/IEC JTC1 SC32 5SQL/OLAP Multi-dimensional Data Model  Sales(prod-id,store-id,time-id,qty,amt) Dimension: Product, Store, Time Hierarchy: – Product -> Category -> Industry – Store->City -> State -> Country – Date -> Month -> Quarter -> Year

ISO/IEC JTC1 SC32 6SQL/OLAP Multi-dimensional Data Model(2)  Operations – roll-up/drill-down – slice/dice – pivot – ranking – comparisons – drill-across – etc.  Example – for each state show me top 10 products based on total sales – what is the percentage growth of Jan-99 total sales over total Jan-98? – for each product show me the quantity shipped and sold

ISO/IEC JTC1 SC32 7SQL/OLAP Database Back in the OLAP Game - History of SQL Evolutions in 1990s(OLAP Area) -  Requirements from industries(‘95 ~ ‘96) – R. Kimball, “Why Decision Support Fails and How to Fix it?” ([2]); see also [3], [4]  Reactions from researchers(‘96) – Jim Gray et al., “Data Cube: A Relational Aggregation Operator Generalizing Group-By, Cross-Tab and Sub Totals,” ([7,8]) – Chatziantoniou, K. Ross, “Querying Multiple Features in Relational Databases,”([9])  Commercial DBMSs and SQL standards(‘98 ~ ) – commercial products: e.g. Oracle, “Analytical Functions for Oracle8i”, Oct., 1999 – SQL standards  ANSI X3H (R3): Super Sets(The Cube and Beyond)  ANSI NCITS H : Introduction to OLAP ­see also [6]

ISO/IEC JTC1 SC32 8SQL/OLAP OLAP Operations  Many business operations was hard or impossible to express in SQL – multiple aggregations – comparisons(with aggregation) – reporting features  Be prepared for serious performance penalty  Client and middle-ware tools provide the necessary functionality – OLAP server: ROLAP vs. MOLAP

ISO/IEC JTC1 SC32 9SQL/OLAP Multiple Aggregations  Create a 2-dimensional spreadsheets that shows sum of sales by maker as well as model of car  Each subtotal requires a separate aggregate query RED WHITE BLUE ChevyFord By Make By Color Sum Cross Tab SELECT color, make, sum(amt) FROM sales GROUP BY color, make union SELECT color, sum(amt) FROM sales GROUP BY color union SELECT make, sum(amt) FROM sales GROUP BY make union SELECT sum(amt) FROM sales

ISO/IEC JTC1 SC32 10SQL/OLAP Comparisons  Examples: – last year’s sales vs. this year’s sales for each product  requires a self-join VIEW: create or replace view v_sales as select prod-id, year, sum(qty) as sale_sum from sales group by prod-id, year; QUERY: select cur.year cur_year, cur.sale_cur_sales, last.sum last_sales from v_sales curr, v_sales last where curr.year=(last.year+1)

ISO/IEC JTC1 SC32 11SQL/OLAP Reporting Features  It was too complex to express – rank(top 10) and N_tile(“top 30%” of all products) – median, mode, … – running total, moving average, cumulative totals

ISO/IEC JTC1 SC32 12SQL/OLAP Reporting Features(2)  Examples: – a moving average(over 3 day window) of total sales for each product for 2000 VIEW: create or replace view v_sales as select prod-id, time-id, sum(qty) as sale_sum from sales group by prod-id, time-id; QUERY: select end.time, avg(start.sale_sum) from v_sales start, v_sales end where end.time >= start.time and end.time <= start.time+2 group by end.time

ISO/IEC JTC1 SC32 13SQL/OLAP OLAP Servers Processing MD queries efficiently

ISO/IEC JTC1 SC32 14SQL/OLAP ROLAP OLAP Client OLAP Engine Relational Database (Star or Snowflake Schema) meta-data To map warehouse schema into a MD model

ISO/IEC JTC1 SC32 15SQL/OLAP ROLAP(2)  Example: Oracle Discoverer 4i leverages Oracle 8i – 8i - biggest SQL improvements in a decade! – more powerful analysis using new analytic functions – sharing query redirection(rewrite) using MVs – 100% automated summary management

ISO/IEC JTC1 SC32 16SQL/OLAP MOLAP  A multidimensional database(MDDB) stores data in a series of array structures, indexed to provide optimal access time to any element in the array.  Example: Oracle Express stores arrays of data

ISO/IEC JTC1 SC32 17SQL/OLAP Propose SQL Constructs  Multiple aggregations – Gray et. al., “Cube and Roll-Up”[6,7]  Comparison – Chatziantoniou and Ross, “Group By Column Variable”[8] SELECT subscriber, r.login-time FROM log GROUP BY subscriber: r SUCH THAT r.spent-time = max(spent_time)  Reporting – Redbrick provides SQL extensions in RISQL  rank, tertile, ratio-to-report etc

ISO/IEC JTC1 SC32 18SQL/OLAP The Data CUBE Relational Operator Generalizes Group By and Aggregates CHEVY FORD RED WHITE BLUE By Color By Make & Color By Make & Year By Color & Year By Make By Year Sum The Data Cube and The Sub-Space Aggregates RED WHITE BLUE ChevyFord By Make By Color Sum Cross Tab RED WHITE BLUE By Color Sum Group By (with total) Sum Aggregate source:[6]

ISO/IEC JTC1 SC32 SQL/OLAP Getting Sub-totals: ROLLUP Operation SELECT year, brand, SUM(qty) FROM sales GROUP BY ROLLUP (year, brand); YEARBRAND SUM(qty) 1996Ford Honda Toyota Ford300 …

ISO/IEC JTC1 SC32 SQL/OLAP Getting Cross-tabs: CUBE Operation SELECT year, brand, SUM(amount) FROM sales GROUP BY CUBE (year, brand); YEARBRAND SUM(AMOUNT) 1996 Ford Toyota Ford Ford 550 Honda 650 Toyota 1000

ISO/IEC JTC1 SC32 SQL/OLAP Flexible Grouping: GROUPING_SETS Operator SELECT year, brand, color, SUM(qty) FROM sales GROUP BY GROUPING_SETS ((year, brand), (brand,color),()); YEARBRAND COLOR SUM(QTY)1996 Ford Honda Toyota Ford Honda Toyota 550 Ford Blue 400 Ford Red 150 Honda Blue 650 Toyota Red 700 Toyota White Brand, Color Year, Brand Grand total

ISO/IEC JTC1 SC32 22SQL/OLAP LAG Operator TIMEKEY SALES SALES_LAST_YEAR SALES_CHANGE ….. … … SQL> SELECT timekey, sales 2 LAG(sales, 12) OVER 3 (ORDER BY timekey) AS sales_last_year, 4 (sales - sales_last_year) AS sales_change 5 FROM sales;

ISO/IEC JTC1 SC32 23SQL/OLAP MOVING Average SELECT time-id, avg(sum(qty)) over (order by time-id RANGE INTERVAL ‘2’ DAY PRECEDING ) as mvg_avg_sales from sales group by time_id ;

ISO/IEC JTC1 SC32 24SQL/OLAP  Why enhance the RDBMS for OLAP calculations? – Performance – Scalability – Simpler SQL development – Productivity

ISO/IEC JTC1 SC32 25SQL/OLAP Database Back in the OLAP Game  Materialized views  Index techniques: e.g. bitmap (join) index  Partitioning: e.g. range/hash/list  Query optimization: e.g. star query optimization ......

ISO/IEC JTC1 SC32 26SQL/OLAP Future OLAP Trends To be or not to be? OLAP API: -OLE DB for OLAP -JOLAP

ISO/IEC JTC1 SC32 27SQL/OLAP References [1] E.F. Codd et al., “Providing OLAP(On-line Analytical Processing) to User-Analysts: An IT Mandate,” Available from Arborsoft’s Web Site( [2] R. Kimball, “Why Decision Support Fails and How to Fix it?” SIGMOD Record, Sep.,1995 [3] R. Kimball, “The Problem with Comparisons,” DBMS Magazine, Jan., 1996(also available from [4] R. Kimball, “SQL Roadblocks and Pitfalls,” DBMS Magazine, Feb., 1996(also available from [5] R. Winter, “Database Back in the OLAP Game,” Intelligent Enterprise Magazine, Dec., 1998,(available from [6] R. Winter, “SQL-99’s New OLAP Functions,” Intelligent Enterprise Magazine, Jan., 2000,(available from [7] Jim Gray et al., “Data Cube: A Relational Aggregation Operator Generalizing Group-By, Cross-Tab and Sub Totals,” Proceedings of International Conferences on Data Engineering, p , 1996 [8] Jim Gray et al., “Data Cube: A Relational Aggregation Operator Generalizing Group-By, Cross-Tab and Sub Totals,” Data Mining and Knowledge Discovery Journal, Vol. 1, No. 1, 1997 [9] D. Chatziantoniou, K. Ross, “Querying Multiple Features in Relational Databases,”, Proc. Of VLDB Conf., 1996