Matthew Stephen – SQL Server Evangelist What's NEW in SQL 2005 for Business Intelligence and Data Warehousing  Matthew Stephen IT Pro Evangelist http://blogs.technet.com/mat_stephen Matthew Stephen – SQL Server Evangelist http://blogs.technet.com/mat_stephen

Agenda BI Platform RDBMS BI Enhancements ETL tool (SQL Server Integration Services) Unified Dimensional model Reporting Services RDBMS BI Enhancements

Business Intelligence Elements of the process Analysis & Planning Client Applications Operational Source Systems Data Warehouse OLAP Cubes Data Marts Operational Data Store Forecasting Vertical LOB Applications SSIS Vertical Industry Solutions Relational Data Warehouse Database Cross Sell Target Sell ERP/ Accounting Systems SSIS MS Reporting Services MS Excel (+ Addin) MS Business Scorecard Manager MS Data Analyse MS Map Point SSIS CRM Internet Commerce Systems Data Mines 3rd Parties Feedback/React/Adjust – Closed Loop

Microsoft BI Platform

SQL Server Integration Services A new Microsoft SQL Server Business Intelligence application The successor to Data Transformation Services The platform for a new generation of high performance data integration technologies

Before Integration Services Alerts & escalation Data mining Call centre data: semi structured Legacy data: binary files Application database Hand coding Staging Text Mining ETL Cleansing & ETL Staging ETL Warehouse Reports Mobile data Integration and warehousing require separate, staged, operations. Preparation of data requires different, often incompatible, tools. Reporting and escalation is a slow process, delaying smart responses. Heavy data volumes make this scenario increasingly unworkable.

With Integration Services Alerts & escalation Warehouse Reports Mobile data SQL Server Integration Services Text mining components Custom source Standard sources Data cleansing Merges Data mining Call centre: semi-structured data Legacy data: binary files Application database Integration and warehousing are a seamless, manageable, operation. Sourced, prepare and load data in a single, auditable process. Reporting and escalation can be parallelized with the warehouse load. Scales to handle heavy and complex data requirements.

Enabling new architectures … Traditional (DTS) warehouse loading Integration process simply conforms data and loads the database server The database performs aggregations, sorting and other operations Database competes for resources from user queries This solution does not scale very well

How SSIS Operates Data sources can be diverse, including custom or scripted adapters Transformation components shape and modify data in many ways. Data is routed by rules or error conditions for cleansing and conforming. Flows can be as complex as your business rules, but highly concurrent. And finally data can be loaded in parallel to many varied destinations.

Enabling new architectures … Warehouse loading with SQL Server Integration Services SQL Server Integration Services conforms the data But also aggregates and sorts, and loads the database This frees-up the database server for user queries

What is the UDM? Enterprise BI - Today Data Source Data Model Tool MOLAP OLAP Browser (2) Browser (1) Datamart Reporting Tool (1) Reporting Tool (1) DW Reporting Tool (3)

Enterprise BI – a Messy Reality Data Source Data Model Tool Duplicate Data Duplicate Models MOLAP OLAP Browser (2) Browser (1) OLAP vs. Reporting Datamart Reporting Tool (1) Reporting Tool (1) DW Reporting Tool (3)

Relational VS OLAP Reports Feature Relational OLAP Flexible schema Real time data access Single data store Simple management Detail reporting High performance End-user oriented Ease of navigation and exploration Rich analytics Rich semantics

The Unified Dimensional Model – The Best of Relational and OLAP Multiple fact tables Full richness the dimensions’ attributes Transaction level access Star, snowflake, 3NF… Complex relationships: Multi-grains, many-to-many, role playing, indirect… Recursive self joins Slowly changing dimensions Relational Reporting Multidimensional navigation Hierarchical presentation Friendly entity names Powerful MDX calculations Central KPI framework “Actions” Language translations Multiple perspectives Partitions Aggregations Distributed sources OLAP Cubes

The Unified Dimensional Model – The Best of Relational and OLAP Multiple fact tables Full richness the dimensions’ attributes Transaction level access Star, snowflake, 3NF… Complex relationships: Multi-grains, many-to-many, role playing, indirect… Recursive self joins Slowly changing dimensions Relational Reporting Multidimensional navigation Hierarchical presentation Friendly entity names Powerful MDX calculations Central KPI framework “Actions” Language translations Multiple perspectives Partitions Aggregations Distributed sources OLAP Cubes The Unified Dimensional Model

Enterprise BI – With A Unified Dimensional Model Data Source Data Model Tool MOLAP OLAP Browser (2) Browser (1) MOLAP UDM Datamart Reporting Tool (1) Reporting Tool (1) DW BI Applications

Enterprise BI – With A Unified Dimensional Model Data Source Data Model Tool MOLAP OLAP Browser (2) Browser (1) MOLAP UDM Datamart Reporting Tool (1) Reporting Tool (1) DW BI Applications A single dimensional model for all OLAP analysis and Relational reporting needs

Analysis Services – Scaleable, High Performance UDM Server Data Source Tool MOLAP OLAP Browser (2) Browser (1) Analysis Services MOLAP UDM Datamart XML/A or ODBO Reporting Tool (1) Reporting Tool (1) DW BI Applications Optimized SQL to all major RDBMS platforms Most scaleable OLAP store OLE DB and XML/A API’s UDM automatically becomes Web Service API supported by all major BI vendors

Analysis Services – Transparent MOLAP Caching Data Source Tool MOLAP OLAP Browser (2) Browser (1) Analysis Services MOLAP UDM Datamart XML/A or ODBO Reporting Tool (1) Reporting Tool (1) DW Cache BI Applications Notifications “Proactive Caching” – Automatic MOLAP cache creation and management MOLAP becomes transparent – no need to manage an OLAP store any more Relational reporting enjoys MOLAP like performance

Analysis Services – Data Marts are Virtualised Data Source Tool OLAP Browser (2) Browser (1) Analysis Services UDM Datamart XML/A or ODBO Reporting Tool (1) Reporting Tool (1) DW Cache BI Applications UDM’s provide “subject area centric” view of the DW “Perspectives” feature allow user/group specific view of the same data High performance ensured by the Proactive Cache

Analysis Services – Streamlined BI Infrastructure Data Source Tool OLAP Browser (2) Browser (1) Analysis Services DW UDM XML/A or ODBO Reporting Tool (1) Reporting Tool (1) Cache BI Applications Unified logical model for both relational and OLAP with superb performance and scalability One data store to manage ensure data consistency and low TCO Rich user experience with many, many of Microsoft and 3rd party tools

Components of a UDM Security End-user Model Calculations Translations Actions KPIs … Calculations Scope(Customer.Country.USA, *); Sales = 2; End Scope; Basic Dimensional Model Cubes and Dimensions Storage/caching policies Linked objects Data Source View Customer Name ID (PK) Age Order Order# (PK) CustomerID DueDate

What is SQL Server Reporting Services? Reporting platform for both traditional and interactive reports Scalable, manageable and embeddable server infrastructure Integrated with SharePoint, Office applications, browser and other familiar tools Single platform and tools for all types of structured data (relational, hierarchical, multidimensional)

Reporting Services BI User Profiles SQL Server Analysis Services Analysts 5-10% of users SQL Server Reporting Services Information Explorers 15-25% of users Information Consumers 65-80% of users

Reporting Lifecycle Reporting Services is an open and extensible platform supporting the authoring, management and delivery of rich, interactive reports to the entire enterprise. Authoring Management Delivery

Report Authoring Reports are defined in Report Definition Language (RDL), a documented XML schema Use Microsoft or 3rd party tools that support RDL Create single reports from multiple data sources (SQL, OLE DB, ODBC, Oracle, and .NET data providers) More info on the RDL spec: www.microsoft.com/sql/reporting

Report Management Report definitions, folders, and resources are published and managed in a reporting web service Managed reports can be executed either on-demand or via schedule and are cached for consistency and performance Scalable & Extensible server architecture

Report Delivery Traditional (paper) and interactive (web) reports On-demand (“pull”) or event-based (“push”) delivery Choose from multiple formats (HTML, Excel, PDF, XML) Deliver reports to almost any device (e-mail, file share, etc.)

XML Web Service Interface Product Architecture Browser Office Custom Application Security Security Services (NT, Passport, Custom) XML Web Service Interface Data Processing Data Sources (SQL, OLE DB, XML/A, ODBC, Oracle, Custom) Report Server Report Processing Delivery Delivery Targets (E-mail, SharePoint, Custom) Rendering Output Formats (HTML, Excel, PDF, Custom) SQL Server Catalog

Report Services SQL 2005 New Features Direct Client Printing (debut in 2000 sp2) End User Sort Multi-Valued Parameters Custom Report Items Visual Studio No longer requires VS to be installed Improved expression editor Analysis Services Query Designer Sharepoint Reporting Web Parts Report Viewer Controls

Report Viewer Controls: Two Modes of Operation Report Server Mode View and interact reports that are centrally managed on a Report Server Local Mode For processing and viewing reports without connecting to a Report Server Can upgrade local mode applications to server mode

RDBMS BI Enhancements Database Snapshots Consume File as ROWSET in TSQL Partitioned Tables Optimistic Concurrency Controls

© 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.