1 COMP 3503 Deductive Modeling with OLAP with Daniel L. Silver Daniel L. Silver

2 Agenda  What is OLAP?  OLAP, MOLAP and ROLAP  OLAP Functionality  Overview of IBM Cognos Insight  OLAP Pros and Cons

3 What is OLAP?

4 On-Line Analytical Processing OLAP  Term coined by E.F. Codd in a document published in 1993 sponsored by Arbor Software Corp (ESSBASE)  In contrast to OLTP and traditional RDBMS  Defined requirements for databases and tools to implement decision support and business intelligence systems.  Has had a significant impact on the database and business software market.

5 OLAP Definition  Online Analytical Processing = OLAP refers to technology that allows users of multidimensional databases to generate on-line descriptive or comparative summaries ("views") of data and other analytic queries.  OLAP facilities should be integrated into enterprise-wide data base systems allow analysts and managers to monitor the performance of the businessallow analysts and managers to monitor the performance of the business e.g. –number of transactions / sales at different locations by product class by timee.g. –number of transactions / sales at different locations by product class by time Courtesy Anders Stjarne

6 Multidimensional Requirements  Example: Sales volume as a function of product, time, and geography. Product Geography Time Dimensions: Product, Geography, Time Measure: ‘ Sales Volume ’ Courtesy Anders Stjarne More than three dimensional data cube is referred to as a hypercube

7 q Deductive Modelling and Analysis Quarter Month Type Customer Line Brand Number Country Branch Sales Rep Quantity Cost Margin Combination 1 Quarter Month Type Customer Line Brand Number Country Branch Sales Rep Quantity Cost Margin Combination 2 When? Time (1997) Who? Customers (Channels) What? Product (Type) Where? Location (Region) Result? Indicator (Revenue) Comprehensive Sales Analysis Courtesy Anders Stjarne

8 On-Line Analytical Processing 12 Rules of an OLAP Environment by E.F. Codd  Multi-dimensional - data-cubes or hypercubes  Transparent access  Navigation aids  Consistent reporting  Client-sever based  Generic dimensionality  Efficient data storage  Multi-user support  Unrestricted cross- dimensional operations  Intuitive data manipulation  Flexible reporting  Unlimited levels of aggregation

9 On-Line Analytical Processing  Strong connection to multi-dimensional database (MDBMS) model  MOLAP  Data-cubes are typically constructed off-line due to time required to build indices  Dimensions, values, and aggregations are limited to that within data-cube  On-line cube development has allowed RDBMS vendors to survive as major players in OLAP market  ROLAP

10 OLAP, MOLAP and ROLAP

11 OLAP Distributed Framework OLAP functions are independent of: Front-end user interfaceFront-end user interface Back-end data storageBack-end data storage Courtesy Anders Stjarne

12 MDBMS  Relational versus Dimensional Data Rwhttp:// Rwhttp:// Rwhttp:// Rw  ROLAP = Representing dimensional data with RDBMS Star SchemaStar Schema o P.html P.html P.html More details:More details: o o igningtheStarSchemaDatabase/tabid/101/Default.aspx igningtheStarSchemaDatabase/tabid/101/Default.aspx igningtheStarSchemaDatabase/tabid/101/Default.aspx

13 MOLAP vs. ROLAP Multidimensional difficulty handling sparcity efficiently difficulty handling sparcity efficiently direct representation of the data “ cube ” direct representation of the data “ cube ” rapid drill down on summary data rapid drill down on summary data proprietary solutions proprietary solutions better performance response better performance response does not scale well to handle large amounts of detail does not scale well to handle large amounts of detail thin client, analytical processing done on server thin client, analytical processing done on server REF: White, “MOLAP vs ROLAP,” (B&A-15) Relational multidimensional view built on a Relational DBMS hampered by the limitations of SQL handles sparcity automatically stores summary and detail data equally easily easy to share common dimensions across DWs scales well using well-developed relational technology depends on efficient processing of STAR joins and indexes analytical processing done on the client (or middle server) Courtesy Anders Stjarne

14 OLAP Functionality

15 On-Line Analytical Processing Deductive Modeling with OLAP Deductive Modeling with OLAP  Model is developed within the users mind as data is explored  Verification or rejection is facilitated by multi-dimensional functions which display data numerically and graphically  Best practices: Determine suspected variable interaction Determine suspected variable interaction Verify/reject model through exploration Verify/reject model through exploration Drill-down to refine model Drill-down to refine model Maintain record of exploratory findings Maintain record of exploratory findings

16 On-Line Analytical Processing Basic OLAP Functionality Basic OLAP Functionality  Dimension selection - slice & dice  Rotation - allows change in perspective  Filtration -value range selection  Hierarchies of aggregation levels drill-downs to lower levels drill-downs to lower levels roll-ups to higher levels roll-ups to higher levels Tremendous tool for decision support and executive information delivery and analysis

17 OLAP - Sample Operations  Roll up: summarize data total sales volume last year by product category by region total sales volume last year by product category by region  Roll down, drill down, drill through: go from higher level summary to lower level summary or detailed data For a particular product category, find the detailed sales data for each salesperson by date For a particular product category, find the detailed sales data for each salesperson by date  Slice and dice: select and project Sales of beverages in the West over the last 6 months Sales of beverages in the West over the last 6 months  Pivot or rotate: change visual dimensions Courtesy Anders Stjarne

18 OLAP and Data Mining  The final results from OLAP exploration can lead to inductive data mining  Data Mining techniques can be applied to the data views and summaries generated by OLAP to provide more in- depth and often more multidimensional knowledge  Data Mining techniques can be considered analytic extension of OLAP

19 q Multi-dimensional Cubes  A cube is a structure that stores data multi-dimensionally and provides: secure data accesssecure data access fast retrieval of data.fast retrieval of data.  Cubes can be distributed across a network or to individual computers.

20 Measures  The numeric (continuous) data that is collected and stored by your organization.  The performance measures used to evaluate your business. Examples: RevenueRevenue CostCost Quantity soldQuantity sold Units on-handUnits on-hand Hours per JobHours per Job Number of callsNumber of calls Defective units.Defective units. q #% Revenue - Cost = Profit Margin Basic Derived

21 q Dimensions and Levels  Dimensions are a broad group of descriptive data about the major aspects of your business.  Levels represent established hierarchy within dimensions. Dimensions Levels When? Date What? Products Where? Locations Years Months Days Line Type Product Region Branch Country Courtesy Anders Stjarne

22 q Levels and Categories A category is a data item that populates a level in a dimension.A category is a data item that populates a level in a dimension. Levels CategoriesDimension Locations RegionCountryBranchEuropeUnited Kingdom London, U.K. Manchester, U.K. Courtesy Anders Stjarne

23 Application Development Process q Plan measures and dimensions Create the cube Obtain the required data Develop the MDBMS model Explore the cube data using Insight Courtesy Anders Stjarne

24 Basic OLAP Operations Selection (Filter) – within the range of a dimension Selection (Filter) – within the range of a dimension Scope – the range on a dimension Scope – the range on a dimension Slice – a two dimensional ‘ page ’ from the cube Slice – a two dimensional ‘ page ’ from the cube Dice – chopping up along the dimensions Dice – chopping up along the dimensions Drill down analysis - to the detail beneath summary data Drill down analysis - to the detail beneath summary data Rollup/ Consolidate Rollup/ Consolidate Rotate (Pivot) – change dimension orientation Rotate (Pivot) – change dimension orientation o Swap rows and columns o Swap on or off o Change nesting order Reach Through – to the source data detail Reach Through – to the source data detail Calculations / Derivation formulas on the measured facts Calculations / Derivation formulas on the measured facts o Ratios, Rankings, etc. o E.g., NetSales = GrossSales – Cost; NetSales = GrossSales*(1 - Margin) REFS: INMON, Building, Ch. 7, p. 243; White, “MOLAP vs ROLAP,” (B&A-15) Courtesy Anders Stjarne

25 Advanced OLAP Operations  Trend analysis - over broad vistas of time handling time series data, time calculationshandling time series data, time calculations  Key ratio indicator measurement and tracking  Comparisons - present to: past, plan, and others competitive market analysiscompetitive market analysis  Problem monitoring - of variables within control limits  Alerts and Event-Driven Agent Processing Courtesy Anders Stjarne

26 OLAP Pros and Cons

27 On-Line Analytical Processing Strengths of OLAP  Powerful visualization ability via GUI  Fast, interactive response times  Analysis of time series  Deductive discovery of clusters/exceptions  Many OLAP products available and integrated to DB products

28 On-Line Analytical Processing Weaknesses of OLAP  Does not handle continuous variables  Does not automatically discover patterns and models  Generation of a complex hypercubes require some training and experience  Hypercube generation and update - MOLAP Vs. ROLAP

29 On-Line Analytical Processing Products and Suppliers Products and Suppliers  of_OLAP_Servers of_OLAP_Servers of_OLAP_Servers

30 Overview of IBM Cognos Insight OLAP Intro: ugczSGNVXlU ugczSGNVXlU ugczSGNVXlU In depth: ?v=bNw89HUHKEk ?v=bNw89HUHKEk ?v=bNw89HUHKEk

31 Tutorial  IBM Cognos Insight

32 THE END