1 1 The IT Perspective: Data Warehousing, Management, and Analytical Structures Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd

Slides:



Advertisements
Similar presentations
Summary and Q&A Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
Advertisements

1 1 Summary and Q&A Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
Introduction to Data Mining Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
Power BI Rafal Lukawiecki Strategic Consultant Project Botticelli Ltd
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered.
1 1 The Big Picture of Business Intelligence: Goals, Concepts, and the Platform Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
MIX 09 4/15/ :14 PM © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered.
1 1 The Big Picture of Business Intelligence: Goals, Concepts, and the Platform Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
1 1 The Knowledge Worker’s Perspective: Self-Service of BI with Microsoft PowerPivot and Office 2010 Rafal Lukawiecki Strategic Consultant, Project Botticelli.
Delivering BI Through Microsoft Office System 2007 Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
Finding Hidden Intelligence with Predictive Analysis of Data Mining Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
1 1 The IT Perspective: Data Warehousing, Management, and Analytical Structures Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
1 1 The Knowledge Worker’s Perspective: Self-Service of BI Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
Microsoft business analytics Power and simplicity.
Business Intelligence Overview Marc Schöni Technical Solution Professional | Business Intelligence Microsoft Switzerland.
Session 1.
Built by Developers for Developers…. © 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names.
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
1. The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki.
Do It Strategically with Microsoft Business Intelligence! Bojan Ciric Strategic Consultant.
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or.
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or.
Welcome to BI Roadshow 2010 John PowerPivot Microsoft
Feature: Customer Combiner and Modifier © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are.
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or.
demo Instance AInstance B Read “7” Write “8”
Do It Strategically with Microsoft Business Intelligence! Bojan Ciric Strategic Consultant.

Turning data into a business advantage Rafal Lukawiecki Strategic Consultant Project Botticelli
customer.
Design and Planning for a BI Project Reeza Ali Architect Microsoft Corporation.
Finding Hidden Intelligence with Predictive Analysis of Data Mining Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
demo © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names.
Advanced (and attractive) analytics Rafal Lukawiecki Strategic Consultant, Project Botticelli
Welcome to BI Roadshow 2010 Germán Díaz Product Marketing Manager Microsoft Spain.
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material.
Aggregating Knowledge in a Data Warehouse and Multidimensional Analysis Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks.
Welcome José Grilo Server and Tools Lead Microsoft Portugal
© 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or.
Welcome My Name Microsoft Xxx Data Mining and Business Intelligence for Enterprises.
Microsoft Solutions for Business Intelligence.

Event Title Event Date. Module 02—Introduction to Dimensional Modeling Techniques Name Title Microsoft Corporation.
Improving Insight and Decision Making Using Microsoft Business Intelligence and SQL Server 2008 Rafal Lukawiecki Strategic Consultant, Project Botticelli.
The Knowledge Worker’s Perspective: Self-Service of BI with Microsoft PowerPivot and Office 2010 Rafal Lukawiecki Strategic Consultant, Project Botticelli.
Aggregating Knowledge in a Data Warehouse and Multidimensional Analysis Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
Entity Based Staging SQL Server 2012 Tyler Graham
Business Intelligence for Project Server/Online
The IT Perspective: Data Warehousing, Management, and Analytical Structures Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
How EMI Music Implemented Master Data Services with Adatis
Delivering BI Through Microsoft Office System 2007
Andi Comisioneru Principal Group Program Manager Microsoft Corporation
Delivering BI Through Microsoft Office 2007 and PerformancePoint Server 2007 Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd
Andi Comisioneru Principal Group Program Manager Microsoft Corporation
Delivering an End to End Business Intelligence Solution
Виктор Хаджийски Катедра “Металургия на желязото и металолеене”
Presentation transcript:

1 1 The IT Perspective: Data Warehousing, Management, and Analytical Structures Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd

2 2 Objectives Explain the basics of: 1.Master Data Management 2.Data Warehousing 3.ETL 4.OLAP/Multidimensional Data The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material presented is not certain and may vary based on several factors. Microsoft makes no warranties, express, implied or statutory, as to the information in this presentation. Portions © 2010 Project Botticelli Ltd & entire material © 2010 Microsoft Corp. Some slides contain quotations from copyrighted materials by other authors, as individually attributed or as already covered by Microsoft Copyright ownerships. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Project Botticelli Ltd as of the date of this presentation. Because Project Botticelli & Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft and Project Botticelli cannot guarantee the accuracy of any information provided after the date of this presentation. Project Botticelli makes no warranties, express, implied or statutory, as to the information in this presentation. E&OE. This seminar is based on a number of sources including a few dozen of Microsoft-owned presentations, used with permission. Thank you to Chris Dial, Tara Seppa, Aydin Gencler, Ivan Kosyakov, Bryan Bredehoeft, Marin Bezic, and Donald Farmer with his entire team for all the support.

3 3

4 4 SQL Services – Why? Install only the ones you need Which? Integration Services Get your data from the world outside (ETL) Analysis Services Cubes, Data Mining, support for PowerPivot on SharePoint Reporting Services DIY Report Builder and traditional “big” reports Master Data Services Quality of critical master data (cities, colours, customers) Database Engine Data warehouse and OLTP relational storage

5 Master Data Management

6 6 MDM Ensures consistency of data across all organisational uses Impacts overall data quality Processes and tools for: Collection, aggregation, matching, distribution, and persistence of master data Consistently Related to Federated Data Management Key to MDM: Modelling

7 7 Why MDM? It’s About Evolution of Enterprise Architecture

8 8 MDM Processes Batched Acquisition from Staging Tables Members, Attributes, Parent-Child Relationships SQL Integration Services Import & Integration Versioning Changes Auditing Compliance Tracking of Instances Modeling Subscription Views Export to: Operational Systems Data Warehouses BI Analytics Reporting Tools Export & Subscription

9 9 Microsoft Master Data Services SQL 2008 R2 Enterprise, Datacenter, Developer Tools: Master Data Manager Primary tool for managing your master data MDS Configuration Manager IT Pro tool MDS Web Service For developers wanting to extend MDS Concepts: Models Entities Attributes Members Hierarchies Collections Versions Database

10 Modelling Master Data Model organises data at highest level Allowing versioning of changes to data There are typically four categories of models: People (Customers, Staff) Places (Geographies, Cities, Countries) Things (Products) Concepts (Accounts, Behaviours, Transactions)

11 Example: Product MDM Model Product (model) Product (entity) Name (free- form attr) Code (free- form attr) Subcategory (domain- based attr) Name (free- form attr) Code (free- form attr) Category (domain- based attr) Name (free- form attr) Code (free- form attr) StandardCost (free-form attr) ListPrice (free-form attr) Photo (file attr)

12 1. Reviewing a Data Model Using Master Data Services

13 Data Warehouse

14 OLE DB ODBC DB2 Oracle XML SQL Server Analysis Services SQL Server Report Server Models SQL Server Data Mining Models SQL Server Integration Services MySAP Hyperion Essbase SAP NetWeaver BI SQL Server Teradata Rich Connectivity Data Providers

15 Star Schema

16 Star Schema Benefits Simple, not-so-normalized model High-performance queries Especially with Star Join Query Optimization Mature and widely supported Low-maintenance

17 Snowflake Dimension Tables Define hierarchies using multiple dimension tables Support fact tables with varying granularity Simplify consolidation of heterogeneous data Potential for slower query performance in relational reporting No difference in performance in Analysis Services database Potential for slower query performance in relational reporting No difference in performance in Analysis Services database

18 Slowly Changing Dimensions Maintain historical context as dimension data changes Three common ways (there are more): Type 1: Overwrite the existing dimension record Type 2: Insert a new ‘versioned’ dimension record Type 3: Track limited history with attributes

19 Integration and ETL

20 Let’s do ETL with SSIS SQL Server Integration Services (SSIS) service SSIS object model Two distinct runtime engines: Control flow Data flow 32-bit and 64-bit editions

21 The Package The basic unit of work, deployment, and execution An organized collection of: Connection managers Control flow components Data flow components Variables Event handlers Configurations Can be designed graphically or built programmatically Saved in XML format to the file system or SQL Server

22 Control Flow Control flow is a process-oriented workflow engine A package contains a single control flow Control flow elements Containers Tasks Precedence constraints Variables

23 Data Flow The Data Flow Task Performs traditional ETL and more Fast and scalable Data Flow Components Extract data from Sources Load data into Destinations Modify data with Transformations Service Paths Connect data flow components Create the pipeline

24 1. Using SQL Server Integration Services for Splitting Data

25 OLAP/Multidimensional Data

26 Cube = Unified Dimensional Model Multidimensional data Combination of measures and dimensions as one conceptual model Measures are sourced from fact tables Dimensions are sourced from dimension tables

27 Hierarchies Benefits View of data at different levels of summarization Path to drill down or drill up Implementation Denormalized star schema dimension Normalized snowflake dimension Self-referencing relationship

28 Hierarchy Defined in Analysis Services Ordered collection of attributes into levels Navigation path through dimensional space Very important to get right! Customers by Geography Country State City Customer Customers by Demographics Marital Gender Customer

29 Measure Group Group of measures with same dimensionality Analogous to a fact table Cube can contain more than one measure group E.g. Sales, Inventory, Finance Defined by dimension relationships

30 Measure Group Dimension

31 1. Using BIDS to Review Dimension Design 2. Cube Design and Functionality

32 Summary As a platform for enterprise Business Intelligence you should consider four services: Data Warehouse (can be relational) Process for Data Management (MDS) Process for Data Integration (ETL) Analysis (OLAP, Data Mining, Columnar) = SQL Server 2008 R2

33 © 2010 Microsoft Corporation & Project Botticelli Ltd. All rights reserved. The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material presented is not certain and may vary based on several factors. Microsoft makes no warranties, express, implied or statutory, as to the information in this presentation. Portions © 2010 Project Botticelli Ltd & entire material © 2010 Microsoft Corp. Some slides contain quotations from copyrighted materials by other authors, as individually attributed or as already covered by Microsoft Copyright ownerships. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Project Botticelli Ltd as of the date of this presentation. Because Project Botticelli & Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft and Project Botticelli cannot guarantee the accuracy of any information provided after the date of this presentation. Project Botticelli makes no warranties, express, implied or statutory, as to the information in this presentation. E&OE.