1. Steve Simon MVP SQL Server BI http://www.infogoldusa.com
A dive into Data Quality Services SQL Server 2014
Boston Ma
September 27 ,2014
Steve Simon MVP SQL Server BI

2. Steve Simon is SQL Server MVP and a Senior Business Intelligence Development Engineer with Atrion Networking Corporation, Providence RI USA. He has been involved with database design and analysis for over 29 years. Steve has presented numerous papers at PASS summits over the years including PASS Europe, in addition to numerous presentations at SQL Saturday events, the Amsterdam and Copenhagen and other local user groups. He a PASS Virtual Chapter Regional Mentor.

4. Our business challenge

5. Lack of conformity
Products, manufacturers and descriptions can be added to the database table in many ways.
Leaves these ’descriptions’ as unreliable for use within query predicates.
With SQL Server 2008 we had data profiling task which advised of a problem, however had no intelligent solutions.

6. OLAP solutions seemed to be the least flexible.
Lack of conformity
Weekly modifications to core data that should be correctly entered the first time.
OLAP solutions seemed to be the least flexible.
Reprocessing of cubes and the time expended can mount up.

7. Enter Data Quality Services

8. What is Data Quality Services?
A set of tools which allow data stewards to improve data quality (Domain experts).
Produces a result set with suggested improvements.
Does not change the original source data set.

9. Why should we use Data Quality Services?
You can get Subject Matter Expert(SME)input.
Manually define, match and cleanse (Man & Machine) .
Computer cleansing of your data. How?
Programmatically, then manually approve. The system learns.

10. Why should we use Data Quality Services?
Can integrate with third party data.
Can integrate with other data processing e.g. SSIS.

11. How to use Data Quality Services
List of basic steps:
Create / Refine / Use a knowledge base.
Perform a data quality evaluation.
Generate output.

12. List of components
DQS Server.
DQS Client.

13. Three main activities

14. When do we use Data Quality Services?
Issue
Detail
Completeness
Is all the data there?
Conformity
Is all the data in the correct format? (capitals ?)
Consistency
Do values represent the same meaning? (Data Mining and Human check)
Accuracy
Do data objects represent real world values?
Validity
Do data values fall within acceptable ranges?
Duplication
Are there multiple copies of the same data?

15. Courtesy Elad Ziklik

16. Requirements SQL Server 2012 Post InstallBI
Edition
Must run DQS Installation Post SQL install
Requirements
Enterprise Edition
Do Master Data Services Integration

17. Installing the server portion

19. After installation in SSMS

20. After having installed the DQS Server portion of the application, you get a few new databases and new security roles.

21. There is no API at present, so we must work with the ‘native’ client.
Designed for single write (at this moment)
There is no API at present, so we must work with the ‘native’ client.

22. Being a single write system..
While editing KB it is locked. Others cannot use.
Complex KB.. Split up and import later.
Each person can create their own entities or domains.
KB ’pieces’ can be imported.

23. Mean while back at the ranch !
‘The challenge’

25. Cleansing Issues
Are duplicates wrong here?

26. Demo

27. SSIS

29. Data quality can be problematic.Human problems.
Take Aways
Data quality can be problematic.Human problems.
DQS like any mining model is able to ’learn’.
DQS learns data patterns and can detect data duplication?
Human intervension decreases as models mature.

30. DQS knowledge bases can be used with SSIS.
Take Aways
Better quality data reduces data processing times. E.g. Cubes effective and efficient.
DQS knowledge bases can be used with SSIS.
Output correct/problematic can be sent to data stewards for validation and verification.
Less frustration from end users and decision makers.

31. At the end of the day it all boils down to
More importantly, why all of us can definitely benefit from a..

32. Steve Simon MVP SQL Server BI http://www.infogoldusa.com
A dive into Data Quality Services SQL Server 2014
Boston Ma
September 27 ,2014
Steve Simon MVP SQL Server BI