1 Lecture 09: OLAP

2 2+2 /* Microsoft SQL Server 2005 */ /* By the way, it is just VHVyaW5nIG1hY2hpbmU= :-) */ WITH SubQuery(t, s, a, b) AS ( SELECT 0, 's', CAST (' 1 */ SELECT '4', '1', '4', '1', 'r' UNION ALL /* find 0 or _; left */ SELECT '4', '_', '5', '_', 'l' UNION ALL SELECT '4', '0', '5', '0', 'l' UNION ALL SELECT '5', '1', '6', '0', 's' UNION ALL /* 1 -> 0 */ SELECT '6', '1', '6', '1', 'l' UNION ALL /* rewind */ SELECT '6', '0', '6', '0', 'l' UNION ALL SELECT '6', '<', 's', '<', 's' /* restart */ ) AS prog(currS, currZ, newS, newZ, mv) WHERE curr.s = currS AND Right(curr.a, 1) = currZ ) SELECT CharIndex('0', a + b) - 2 FROM SubQuery WHERE s = 'a' OPTION (MAXRECURSION 0); /* SELECT t, s, a + '.' + b FROM SubQuery OPTION (MAXRECURSION 0); */ David Srbecky

3 Acknowledgments DB2/400: Mastering Data Warehousing Functions. (IBM Redbook) Chapters 1 & 2 only. Data Warehousing and OLAP Hector Garcia-Molina (Stanford University) Data Warehousing and OLAP Technology for Data Mining Department of Computing London Metropolitan University

4 Buzz Words Buzz Words Buzz Words Buzz Words Buzz Words Data Warehouse (DW) Decision Support (DS) Data Marts (DM) Data Mining (DM) Enterprise Dashboard (ED) Multi-Dimensional Modeling (MDM) Online Analytic Processing (OLAP) Extract, Transform, and Load (ETL) MOLAP vs. ROLAP Three Letter Acronym (TLR) Drill Down, Roll up (DD+RU) Data vs. Knowledge (DvK) Data Cube vs. Sugar Cube (DCvSC) Don’t be surprised to see this sort of BDB (Blah-Dee-Blah) in the trade press: “The ED lets you transform enterprise data into knowledge with at-a-glance DS/DM and MDM, allowing interactive DD/RU over large DCs.”

5 OLTP vs. OLAP Database is operational Data is up-to-date Mostly updates Need to support high levels of update transactions Normal form schemas are important Database is for analysis Data is historical Mostly reads Need to efficiently support complex queries, and only bulk loading of data Schema optimized for query processing

6 Decision Support Systems Information SourcesData Warehouse Server (Tier 1) OLAP Servers (Tier 2) Clients (Tier 3) Operational DB’s Semistructured Sources Extract Transform Load Data Marts Data Warehouse e.g., MOLAP e.g., ROLAP serve Analysis Query/Reporting Data Mining serve From Enrico Franconi CS 636

7 xOLAP Multi-dimensional OLAP (MOLAP) –‘A k-dimensional matrix based on a non relational storage structure.’ [Agrawal et al] Relational OLAP (ROLAP) –‘A relational back-end wherein operations of the data are translated to relational queries.’ [Agrawal et al] Hybrid OLAP (HOLAP) –Integration of MOLAP with ROLAP. Desktop OLAP (DOLAP) –Simplified versions of MOLAP or ROLAP. ZOLAP –Speak with your chemist (normally only prescribed for death march victims)

8 Beware of Data Warehouse Death March Edward Yourdon, 1997, Death March: The Complete Software Developer’s Guide to Surviving “Mission Impossible Projects” Death March projects “use a forced march imposed upon relatively innocent victims, the outcome of which is usually a high casualty rate.” Data Warehouses and Decision Support systems are among the most complex and demanding in the IT world. Failure rates are very high….

9 Relational data model based on a single structure of data values in a two dimensional table CUSTOMER ORDER Cus_idCus_name… 001Robert… 002Lyn… ……… Ord_noOrd_dateCus_id… 0102 Dec 02002… 0203 Dec 02Lyn… …………

10 Data warehousing ___Multidimensional Data Sales volume as a function of product, month, and region Product Region Dimensions: Product, Location, Time Month

11 A Sample Data Cube Total annual sales of TV in U.S.A. Date Product Country sum TV VCR PC 1Qtr 2Qtr 3Qtr 4Qtr U.S.A Canada Mexico sum

12 A Concept Hierarchy for Dimension Location all EuropeNorth_America MexicoCanadaSpainGermany Vancouver M. WindL. Chan... all region office country TorontoFrankfurtcity

13 Cuboids Corresponding to the Cube all product date country product,dateproduct,countrydate, country product, date, country 0-D(apex) cuboid 1-D cuboids 2-D cuboids 3-D(base) cuboid

14 Multidimensional Data: A University Sample Data Cube Students’ marks as a function of student, department, and year Average Mark of Abraham in Year 1. Module Student Time Avg Abraham Caroline Bridget Art Business Computing Year 1 Year 2 Year 3 Avg Design

15 Data Warehousing “A data warehouse is a subject-oriented, integrated, time-variant, and nonvolatile collection of data in support of management’s decision-making process.” —W. H. Inmon

16 OLAP Operations Roll up (drill-up): summarize data –by climbing up hierarchy or by dimension reduction Drill down (roll down): reverse of roll-up –from higher level summary to lower level summary or detailed data, or introducing new dimensions Slice and dice: –project and select Pivot (rotate): –reorient the cube, visualization, 3D to series of 2D planes. Other operations –drill across: involving (across) more than one fact table –drill through: through the bottom level of the cube to its back- end relational tables (using SQL)