Data Warehousing & Business Intelligence Introduction What do you think of when you hear the words Data Warehousing ? Prithwis Mukerjee, Ph.D.

2 © Prithwis Mukerjee Conceptual DW Definition Data warehousing is a program dedicated to the delivery of information which advances decision making, improves business practices, and empowers workers.

3 © Prithwis Mukerjee Data Structure Technology Infrastructure Management Business Applications Information Technology Process People The Knowledge Management Framework

4 © Prithwis Mukerjee Database How it all fits in.. CRM : Customer Relationship Management Transactional Systems ERP : Enterprise Resource Planning SCM : Supply Chain Management Data Warehouse

5 © Prithwis Mukerjee Target Advertising campaigns Strategic Initiatives Business Processes Functions Profitability Analysis Market Basket Analysis Product Pricing Cross-selling and upgrade selling Just-in-Time Inventory Category Management Human Resources Management Determine Customer Lifetime Value Predict Customer Behavior Management Reporting Customer Acquisition and Retention Typical Business Uses of the Data Warehouse

6 © Prithwis Mukerjee Benefits of the Data Warehouse Program Improves the way we do business and the bottom line Revenue Stimulation & Revenue Protection Cost Reduction and Cost Avoidance Productivity Improvement Profitability Enhancement Performance Analysis Decision Making Market Response Competitive advantage

7 © Prithwis Mukerjee DSSs,Report writers, Excel, databases, etc. Data Feeds Budgeting Analysis Non-integrated Decision Support Architecture Inventory System Order System Procurement System Accounting System Sales Forecasting

8 © Prithwis Mukerjee Enterprise DW/ODS Subject oriented Data Warehouses or Data Marts One Stop Data Shopping Basic Data Warehouse Architecture Fewer Data Feeds Inventory System Order System Procurement System Accounting System

9 © Prithwis Mukerjee Performance Measures : Definition & Examples Carefully selected set of measures derived from strategies, goals and objectives that represents a tool to communicating strategic direction to the organization for motivating change. These form the basis to plan, budget, structure the organization and to control results. Innovation & Learning Measures Customer Measures Financial Measures Internal Process Measures  % Sales of New Products  Customers Acquired  Customer Satisfaction  Market Share  ROI and ROA  Revenue Growth  Product Time to Market  Unit Manufacturing Cost  Days Supply to inventory  New Product Introduction  Management Skills  Employee Turnover

10 © Prithwis Mukerjee Differences between OLTP and DW Data Access, Manipulation and Use Data Organisation and Integration Time Handling Usage Data Structures and Schemas Explanations..

11 © Prithwis Mukerjee Data access, manipulation and use Data Entry Transaction Oriented Consistent use patterns Data retrievals are lookups of single records Users deal with one record at a time Performance is critical Reporting is generally table lists Data Query Bulk data oriented Spiked, uneven use patterns Queries are unpredictable, they change continuously Data retrievals are summary and sorts of millions of records Performance is relaxed (sec/min) Reporting is primary activity (on line, presented in small chunks) OLTP DW Differences between OLTP and DW

12 © Prithwis Mukerjee Data Organisation And integration Organized around applications Unintegrated data Different key structures Different naming conventions Different file formats Organized around subject areas Integrated data Standardized key structures Standardized naming conventions Standardized file formats OLTP DW Differences between OLTP and DW

13 © Prithwis Mukerjee Time Handling No time series analysis Data relationships constantly change Changes are instantaneous Limited history, days Twinkling Database …. Time series analysis Data is static over time Series of data snapshots Snapshots create historical database, often greater than two years Quiet database OLTP DW Differences between OLTP and DW

14 © Prithwis Mukerjee Usage Place an order for a product Look up price for a product Apply discount Assign shipper Trigger inventory pick-list Verify shipment of product Create invoice for the product Apply credit to sales representative Essential to RUN the company What type of customers are ordering this product? Who are my top 10% accounts? By name, by revenue, by profitability, by region? How are these different by customer segments? By sales rep? By store? Which shippers have the best on time delivery records ? How does this vary by shipment size? By season of year? Essential to WATCH the company OLTP DW Differences between OLTP and DW

15 © Prithwis Mukerjee Data Structures & Schemas Drives out all data redundancy Improves performance Divides data into many discrete entities Tables are symmetrical Can ’ t tell most important, largest, which hold measures, which are static descriptors Lots of connection paths between tables prefers to use tables individually or in pairs Too complex for users to understand Data redundancy is encouraged Improves table browsing Subject area oriented. Groups data into categories of business measure and characteristics Tables are symmetrical Large dominant tables Clearly defined connection paths for table joins Simple for users to understand and navigate OLTP DW Differences between OLTP and DW

16 © Prithwis Mukerjee Basic Datawarehousing Topics The Four Building Blocks DW Definition DW Usage and Benefits DW Vs. the non-integrated DSS environment Performance Measures Dimensional Modeling Technical Infrastructure Knowledge Mgmt. Architecture IT and Business Perspectives DW Methodology

17 © Prithwis Mukerjee Dimensional Data Modeling Dimensional Data Modeling techniques organize the content of the data warehouse. It structures the data according to the way users ask business questions.

18 © Prithwis Mukerjee The Technical Infrastructure A technical infrastructure provides the physical framework to support data acquisition, storage, access, and data management. It involves development and integration of hardware and software components.

19 © Prithwis Mukerjee Metadata Source Data Purchasing Systems General Ledger Other Internal Systems External Data Sources Data Resource Management And Quality Assurance. Invoicing Systems Data Extraction Integration and Cleansing Processes Extract ODS Purchasing Marketing and Sales Corporate information Product Line Location Translate Attribute Calculate Derive Summarize Synchronize Segmented Data Subsets Summarized Data Data Warehouse Applications Custom Developed Applications Data Mining Statistical Packages Query Access Tools Data Marts Transform Knowledge Management Architecture

20 © Prithwis Mukerjee The Business and The IT Perspective Business What will it do? What value will it bring? How is it built? How does it work? Information Technology Data Warehouse

21 © Prithwis Mukerjee The Business Perspective of the Data Warehouse It takes forever to get the information I need to do my job When I do get it, it ’ s wrong We have mountains of data, but I can ’ t figure out what ’ s important It takes so long to get the data that I don ’ t have any time left over to analyze it I want it to be easy. Just let me point and click my way to an answer I want to see my data in every possible combination Data is scattered everywhere across our organization. Where do I look ? I want a historical view of the business I want to predict the future Focuses on needs and usage

22 © Prithwis Mukerjee The IT Perspective of the Data Warehouse Organizes and stores data by subject area rather than application Extracts and integrates data from multiple source systems into a single database Provides data cleansing, summarization, and calculation User does not create, update, or delete data Provides snapshots of data over periods of time Supports analytical processing, not transactional processing Builds a technology infrastructure to support data acquisition, data storage, data access, and metadata capture Focuses on database, technology, organizational features

23 © Prithwis Mukerjee DW Methodology The methodology provides a detailed roadmap to organize and perform the tasks required in building the data warehouse

24 © Prithwis Mukerjee Data Warehouse System Development Life Cycle CONSTRUC- TION IMPLEMEN- TATION DESIGN ANALYSI S PLANNING MANAGING Business Architecture Data Architecture Technology Architecture Management Infrastructure

25 © Prithwis Mukerjee stopstop