Mastering Master Data Services Presented By: Jeff Prom BI Data Architect Bridgepoint Education MCTS - Business Intelligence, Admin, Developer

Please Support Our Sponsors SQL Saturday is made possible with the generous support of these sponsors. You can support them by opting-in and visiting them in the sponsor area.

Orange County User Group Local User Groups Orange County User Group 2rd Thursday of each month bigpass.pass.org Los Angeles User Group San Diego User Group 1st & 3rd Thursday of each month 3rd Thursday of each odd month meetup.com/sdsqlug sql.la meetup.com/sdsqlbig Malibu User Group Los Angeles - Korean 3rd Wednesday of each month Every Other Tuesday sqlmalibu.pass.org sqlangeles.pass.org

Agenda What is Master Data Services? What is Master Data Management? Installation & Configuration MDS Modeling Concepts Features & Functionality Excel Add-in Demo’s Throughout Questions / Wrap up

Master Data Management What Is? Master Data Services Master Data Management 10 | 43

What is Master Data Services? Microsoft's solution to Master Data Management Traditionally MDM solutions have been expensive, complex, and focused on a single domain. Domain = Group of related data in a focused area. MDS is flexible and can be used with multiple domains. It is relatively inexpensive and easy to use. 11 | 43

What is Master Data Management? Master Data Management (MDM) The management of non-transactional data. Managing relationships between data groups. Managing hierarchies within data groups. Empower data stewards, not IT, to ensure data quality. Provide an accurate source of data for other systems. Data security. Even though Master Data manages non-transactional data, it is almost always used within transactions. Data Stewards help manage the data. Technically savvy Subject matter experts A common example of poor MDM is the scenario of a bank at which a customer has taken out a mortgage and the bank begins to send mortgage solicitations to that customer, ignoring the fact that the person already has a mortgage account relationship with the bank. This happens because the customer information used by the marketing section within the bank lacks integration with the customer information used by the customer services section of the bank. Thus the two groups remain unaware that an existing customer is also considered a sales lead. MDM does not manage numeric data. However, all transactions usually involve customer, products, etc. 12 | 43

Why do I need MDM? One source of data High Data Quality Reduce redundant data sources Eliminate wasted time and effort managing multiple datasets of the same data High Data Quality Leverage Data Stewards Data validation Data is up to date and accurate Quickly and easily manage data changes 13 | 43

Integrating MDM with a Data Warehouse

Questions? 17 | 43

Master Data Services Let’s Dig In… 18 | 43

MDS Background Version 1.0 in SQL Server 2008 R2 June, 2007 - Microsoft acquires Atlanta based Stratature Stratature +EDM product becomes MDS. Many Stratature employees moved on to a company called Profisee which is also based in Atlanta. Version 1.0 in SQL Server 2008 R2 Available in Enterprise & Business Intelligence Editions http://en.wikipedia.org/wiki/Microsoft_SQL_Server_Master_Data_Services 19 | 43

How do you use MDS? MDS User Interface Web Based Full functionality Data Management Excel Add in (New in SQL Server 2012) Limited MDS functionality Very useful when working with data 20 | 43

MDS Data Flow Add SSIS. Add multidirectional arrows and notes? 21 | 43

Installation and Configuration 1. Install Prerequisites PowerShell IIS .NET Framework 2. Create a new database 3. Web Configuration Website & Web Application Associate to the MDS database Enable DQS Integration (optional) Enables Matching Functionality In the Master Data Services Add-in for Excel, matching functionality is provided by Data Quality Services (DQS). This functionality must be enabled to be used. 22 | 43

Install & Configuration 30 | 43

MDS Modeling Concepts Models (Database) Highest level of organization. Containers of related entities. Entities (Table) Base containers for data. Attributes (Columns) Attributes that describe members. Members (Records) Records that populate the entities. P. 56 24 | 43

Models (Databases) Highest level of data organization Models contain the following objects: Entities Attributes and attribute groups Explicit and derived hierarchies Collections Hierarchies and collections can be used to group members for reporting or analysis. White = Model 25 | 43

Entities (Tables) Contained in MDS models Groupings of similar data Contains members Typically one or more related entities per model The dark blue are examples of entities. 26 | 43

Attributes (Columns) & Attribute Groups Attribute values describe the members of the entity. Three types of attributes: Free-form Domain-based File attributes Attribute Groups Organize attributes in an entity Web App shows group name instead of many attributes Easily set security on groups Free-form: Allow free-form input for text, numbers, dates, or links. Domain-based: An attribute with values that are populated by members from another entity. (a constrained list) File attributes: Used to store files, documents, or images. File attributes are intended to help with the consistency of your data by requiring files to have a specific extension. File attributes cannot be guaranteed to prevent a malicious user from uploading a file of a different type.

Members (Records/Rows) Members are the physical master data. Three types of members: Leaf (default) Consolidated (Exist only when explicit hierarchies and collections are enabled for the entity) Collection Leaf members are the default members in an entity. In derived hierarchies, leaf members are the only type of member. Leaf members from one entity are used as parents of leaf members from another entity. In explicit hierarchies, leaf members are the lowest level and cannot have children. Consolidated members exist only when explicit hierarchies and collections are enabled for the entity. Derived hierarchies do not contain consolidated members. In explicit hierarchies, consolidated members can be parents of other members within the hierarchy, or they can be children. 28 | 43

Hierarchies Derived Explicit Recursive Derived with an Explicit Cap Deprecated in 2016 Recursive Derived with an Explicit Cap https://msdn.microsoft.com/en-us/library/ee633737.aspx 32 | 43

Excel Add-in New in SQL 2012. Updated in 2016 Office 2007 or higher 32-bit and 64-bit available Good for working with 1million records or less Has most functionality, except: Business Rules New Models Hierarchies (Unless using the 2016 Add-in) MDS Excel Add-In for Office 2013: http://www.microsoft.com/en-us/download/details.aspx?id=35581 29 | 43

Using MDS – The Basics 30 | 43

Collections Deprecated in 2016 Leaf and consolidated members from a single entity A flat list of members No hierarchy Can contain other collections https://msdn.microsoft.com/en-us/library/ee633733.aspx 33 | 43

Business Rules If/Then statements that result in an action Rules help ensure the quality and accuracy of the data Possible actions: Default Value Change Value Validation Email External Action 34 | 43

Staging MDS Good for working with over 1 million records or automated processes At least one table is created per entity Use the staging process to: Create, update, deactivate, and delete leaf and consolidate members. Update attribute values. Designate relationships in explicit hierarchies. Load data into the staging tables. stg.name_Leaf stg.name_Consolidated stg.name_Relationship Use stored procedures, or the MDS site, to load data from staging tables into MDS. stg.udp_name_Leaf stg.udp_name_Consolidated stg.udp_name_Relationship Yes, it actually generates custom stored procedures per entity which can be used. https://docs.microsoft.com/en-us/sql/master-data-services/overview-importing-data-from-tables-master-data-services 35 | 43

Publishing Data Create subscription Views in MDS Regenerate the view if a model changes Views work with: Leaf members Consolidated members Collections Hierarchies (Explicit and Derived) https://msdn.microsoft.com/en-us/library/ee633741.aspx 36 | 43

Versions & Flags Versions allow you to create copies of the master data within a model. Can be locked while you validate your data, and then committed after the data has been validated. Use versions to: Create an auditable record of your data as it changes over time. Prevent changes while you ensure data validates against business rules. Lock down a model for use by subscribing systems. Test different hierarchies without implementing them right away. Use flags to mark and identify versions, such as ‘Latest’. https://msdn.microsoft.com/en-us/library/hh231030.aspx 37 | 43

Security Broad range of security options Able to specify security down to individual attributes. Based on local or Active Directory users and groups. (2014) Read-only, Update, Deny (2016) Read, Create, Update, Delete, Deny, Admin, Clear 2014 and before. Only 1 Admin User (AD Group’s do not work) EXEC [mdm].[udpSecuritySetAdministrator] @UserName='DOMAIN\user_name', @SID='SID', @PromoteNonAdmin=1 2016 Allows more than 1 admin (Super User) & works with AD Groups Use the Web App You can actually make an AD group an admin but it won’t work properly or as expected. https://msdn.microsoft.com/en-us/library/hh231026.aspx 38 | 43

Notifications Configure to send email notifications when: The status of a model version changes When business rule validation fails The status of a changeset changes Configure mail in the MDS Configuration Tool https://msdn.microsoft.com/en-us/library/ff487053.aspx 26 | 43

Deployments Model Package Editor (ModelPackageEditor.exe) Use to promote through Dev, QA, Prod Model Deployment Wizard (Web App) Model only Copy data manually, use SSIS, or staging MDSModelDeploy tool (MDSModelDeploy.exe) Model & Data Command prompt Model Package Editor (ModelPackageEditor.exe) Wizard tool to edit a model package Subscription views need to be created manually https://msdn.microsoft.com/en-us/library/ff486956.aspx

Change Tracking Use change tracking groups to take action when an attribute value changes. Use change tracking when you don’t know what the new value will be, but instead want to know if any change occurred. To configure change tracking, you add an attribute to a change tracking group. The group can contain one or many attributes. Then, you create a business rule to define the action that is taken when any of the attributes in the group change. https://msdn.microsoft.com/en-us/library/hh479624.aspx 26 | 43

Transactions A transaction is recorded each time action is taken on a member. Transactions can be viewed by all users and reversed by administrators. Transactions show the date, time, and user who took the action, along with other details. Users can add an annotation to a transaction, to indicate why a transaction took place. In the Explorer functional area, you can view and annotate (add comments to) the transactions that you made yourself. In the Version Management functional area, administrators can view all transactions for all users for the models they have access to, and reverse any of these transactions. https://msdn.microsoft.com/en-us/library/ff714019.aspx 26 | 43

Annotations Annotations are comments that you enter to provide details about transactions. You can annotate a transaction to provide information about why an action was taken. You must select a transaction to add a related annotation. You cannot delete annotations. https://msdn.microsoft.com/en-us/library/ff714021.aspx 26 | 43

What’s New in 2016?

What’s New in 2016? Mostly HTML5 vs Silverlight Improved Performance Improved Security Transaction Log Maintenance Improved Troubleshooting Improved Manageability Custom Indexes Entity Sync Relationship Member Revision History Manage Business Rules (MDS Add-in for Excel) Many-to-Many Relationships in Derived Hierarchies Merge Conflicts https://msdn.microsoft.com/en-us/library/ff929136.aspx

What’s New in 2016? Business Rules Extension Attribute Filters Change Sets Purge Soft Deleted Members Change Set Email and Management Edit Many-to-Many Relationships in Derived Hierarchies View and Manage Revision History Save or Open Query Files in MDS Add-in for Excel Business Rule Management Experience Redesigned Business Rule Management Functionality Removed from the MDS Add-in for Excel Derived Hierarchy Management Experience Improved https://msdn.microsoft.com/en-us/library/ff929136.aspx

Changesets New in 2016 Save pending changes to an entity as changesets Submit the changeset to the entity admin for approval The entity admin gets an email notification that a changeset is waiting for approval If approved, the changeset will be committed automatically If rejected, the changeset is sent back to the owner https://msdn.microsoft.com/en-us/library/mt607159.aspx 26 | 43

Entity Sync New in 2016 One-way and repeatable synchronization between entity versions Enables you to share entity data between models One model can be the ‘master’ source, and then reuse in other models https://msdn.microsoft.com/en-us/library/mt244806.aspx 26 | 43

Using MDS – Advanced 30 | 43

Links Microsoft Installation Guide Microsoft MDS Forums Microsoft MDS Overview https://msdn.microsoft.com/en-us/library/ee633763.aspx Microsoft Installation Guide https://msdn.microsoft.com/en-us/library/ee633752.aspx Microsoft MDS Blog https://blogs.msdn.microsoft.com/mds/ Microsoft MDS Forums https://social.technet.microsoft.com/Forums/sqlserver/en-US/home?forum=sqlmds Microsoft Instructional Videos https://technet.microsoft.com/en-us/dn912438#master-data MDS Excel Add-In for Office 2013: http://www.microsoft.com/en-us/download/details.aspx?id=35581

MDS Books Free eBook with a chapter on MDS! http://go.microsoft.com/FWLink/?Linkid=229829

Thank You! Questions? Jeff Prom Blog: http://jeffprom.com Email: jeffprom@gmail.com LinkedIn: www.linkedin.com/in/JeffProm