06 | Managing Enterprise Data Richard Currey | Senior Technical Trainer–New Horizons United George Squillace | Senior Technical Trainer–New Horizons Great Lakes

Module 6 Overview Master Data Services Data Quality Services

Topic: Master Data Services

Topic: Master Data Services What Is Master Data Services? Data Models Entities and Attributes Hierarchies Business Rules Consuming Master Data

What Is Master Data Services? The SQL Server solution developed to help manage data at an enterprise level Targeted at data that is shared by multiple databases Allows an organization to implement centralized storage Relies on ETL to manage the collection and distribution of the data between instances Uses data models and business rules to define the master data

Data Models A data model is a collection of entities, attributes, and hierarchies Master Data Services can have several data models Data models should represent specific areas of the business Customers Model Version 1 Version 2 Version 3 Account Type Entity Customer Entity Attributes: Code (string) Name (string) Code: 1 Name: Standard Member Code: 2 Name: Premier Code: 1235 Name: Ben Smith Address: 1 High St, Seattle Phone: 555-12345 AccountType: 1 CreditLimit: 1000 Code (free-form text) Name (free-form text) Address (free-form text) Phone (free-form text) AccountType (domain-based) CreditLimit (free-form number) Contact Details Attribute Group

Entities and Attributes Each entity represents a business object that can appear in one or more systems Entities contain attributes that are the values that need to be known about an entity Entities have members that are the instances of an entity Entities typically map to tables in an RDBMS, with attributes mapping to columns Members typically map to rows in the table

Hierarchies Represent relationships between entities or entity members Derived hierarchies represent the relationship between members Based on attribute values Explicit hierarchies represent the relationship between individual members of an entity e.g. all European customers Used when there is no other entity to relate the members together Mandatory hierarchies require all members of the entity to be assigned Derived Hierarchy Code: 1 Name: Standard Account Type Member Code: 1235 Name: Ben Smith Account Type: 1 Customer Member Code: 1267 Name: Amy Strande Explicit Hierarchy Code: CustUS Name: US Customers Consolidated Member Code: 1235 Name: Ben Smith Leaf Member Code: 1267 Name: Amy Strande Code: CustEU Name: European Customers Code: 2214 Name: Sabina Schütz

Creating A Data Model

Business Rules What is the biggest myth in IT? “My data is clean” Ensure that data is valid and correct Validated in Microsoft Excel or in the MDS web interface

Implementing Business Rules

(e.g. Data Warehouse ETL) Consuming Master Data Master data architecture is typically a hub and spoke architecture Data is originally entered into operational systems Data is then transferred to the MDS database Data is validated and cleansed in the MDS database MDS data is then used to update the operational systems CRM Marketing System Order Processing System Master Data Hub  Data Steward Other consumers (e.g. Data Warehouse ETL) SSIS

Topic: Data Quality Services

Topic: Data Quality Services What Is Data Quality Services? Building a Data Quality Services Solution Accessing Reference Data Validating Data with Data Quality Services Validating Data with SSIS

What Is Data Quality Services? Data is the lifeblood of any organization Incorrect data can lead to incorrect decision making Data Quality Services (DQS) implements the tools to ensure that data is correct Knowledge bases Domains Reference data services 1011000110 KB DQS Client Data Cleansing Transformation SSIS DQS Server

Building a Data Quality Services Solution Create a knowledge base Provides access to specific areas of information Create domains Provides a repository for the rules relating to specific data Define rules relating to valid or invalid values Define matching policies to determine if two values represent the same logical entity

Implementing Data Quality Services

Accessing Reference Data Reference data provides values that are used as “correct” during a cleansing project Reference data can originate from inside or outside the organization The Widows Azure Marketplace DataMarket is one external source of reference data

Validating Data with Data Quality Services Data cleansing projects allow data stewards to validate and correct data Projects require a knowledge base and a data source Cleansing can include validation, matching, or both Result includes the original data, cleansed values, reason, status, and confidence

Creating a Data Cleansing Project

Validating Data with SSIS Create a data cleansing project Define data source to be cleansed Configure a data cleansing transform Specify the knowledge base Map data columns Define data destination or subsequent transformations

Configuring the Data Cleansing Transform