Data Warehousing/Loading the DW—Topics Review of Steps & Components SQL Server Integration Services Define Data Sources Create Packages & Process Flows Data Flows for Dimension Tables Data Flows for Fact Tables Lookup Translations Importing Excel Data

Steps and Components Source Systems—provide raw data to the DW Integration Services—Provide transformation and loading services from source data to DW Data Warehouse—Customized data store for Business Intelligence Analysis Services—Tools for data mining and reporting Reporting Services—Out old friend acting on an enhanced data store

Star Schema & Dimension Tables Dimension Tables represent concepts (entities) used to group data in the fact tables Also contain descriptive attributes of the entity represented by the dimension table Simplest way for nontechnical users to picture the data Relate to FKs in the fact tables

Design Steps It is impractical to design a one-source DW as the first deliverable Identify initial scope of DW Problem Statement Business Requirements Build DW Data Model Business Processes to address requirements Level of Detail Fact Tables (what we are measuring) Dimension Tables (how we look at the data)

SQL Server Integration Services SSIS creates projects containing packages to select, transform, and load data From a data source To a data destination (DW) Start projects through Business Intelligence Studio (Same as Reporting Services designers) Packages can be narrowly or broadly focused Recommend narrowly focused packages to do one specific task

Create New Integration Services Project

Create Data Sources Right-Click Data Sources in Solution Explorer Create data sources for source and destination DB

Define Process Flow Process flow defines steps needed to perform package tasks Process flow tasks are found in the toolbar Add a data flow task to the process flow Set name property in properties Define Data flow task by double-clicking Process flows can be simple or incredibly complex

Define Data Flow Middle of Design Area Add source and destination data sources to Connection Managers area—Right-click Select New connection from data source… Select all sources needed for current transformation Source DB Destination (DW) DB

Define Data Flow (cont.) Data flows start with sources that provide columns of data End with an output destination Along the way… Columns can be added Other sources Calculations Columns can be transformed Columns can be ignored Flow columns map to columns in destination table

Adding Data Source & Destination Add Data Source & Destination controls from toolbox OLEDB Data Source & Destination for SQL Server Several other source and destination types available Set Name properties

Configure Data Source Control Double-click Choose avail. Connection Manager Select Data access mode Build queries if needed Use view or SP Use table

Configure Data Destination Control Drag green arrow from source control to destination control Set connection manager to DW Set name of table or view to DW destination table Select Mappings

Configure Data Destination Control (cont.) Confirm column mappings from source to dest. Drag source col onto dest. col if necessary

Modify Process Flow New data is appended to destination table If we want to empty the destination table we can add an Execute SQL Task control Set SQL statement to a DELETE statement

Loading Fact Tables Lookup Translations Fact table loading similar to dimension table loading More likely to be based on a query instead of a table Needs Lookup translations to map from available FKs in source DB to Dimension keys in destination table Lookup Translations

Lookup Translations Add Lookup control to data flow from toolbox Sequence is (usually) not important Set table on which to perform lookup action Use Columns tab to set lookup

Lookup Translations (cont.) Confirm column mappings for lookup values Select value to be returned Select Replace PK if available Otherwise select <add as new column>

Importing Excel Data See government statistical data in excel format http://www.gpoaccess.gov/eop/tables06.html Create new package as before Add Excel Source Set name property

Importing Excel Data (cont.) Double-click Excel source to configure New connection

Importing Excel Data (cont.) Find Excel file Set version Decide whether layout supports providing column names

Importing Excel Data (cont.) Excel data is likely to need some transformations Unusual data typing in Excel Unpivoting pivot tables Extracting data I recommend getting rid of messy data in Excel source file if possible before importing Derived Column Control Data Type Conversion

Importing Excel Data (cont.) As with other data flows the results are a set of columns Final step is to map the developed set of columns to the destination data source columns

Importing Data--Summary There are hundreds (or thousands) of combinations of data sources and transformations to result in data for DW Topics not covered Validating data Many different transformations Using alternative data sources Easy loads are easy Difficult transformations are more challenging