1. Azure Data Catalog Adoption Patterns and Best Practices
11/7/2018
Azure Data Catalog Adoption Patterns and Best Practices
Matthew Roche
Senior Program Manager
Microsoft Corporation
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. Why Azure Data Catalog? 11/7/2018
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

3. 11/7/2018
What are the root problems enterprise organizations are trying to solve?
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

4. Discovery and Democratization
11/7/2018
Discovery and Democratization
Data governance and data source discovery are means to an end, never an end unto itself
Organizations are trying to evolve and change, and this involves data
Success depends on the organization’s motivations and goals, independent of technology
When a organization’s goals include discovery and democratization, Data Catalog may be part of the solution
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

5. 11/7/2018
Key Learnings
Having data available is no longer sufficient – data must be known and understood in order to provide strategic value
The conversation is no longer about technology – it is about the goals of the organization, the key scenarios, and the challenges faced in achieving them
Governance requires a change to culture, not simply new tools and technologies
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

6. 11/7/2018
Problem Domain
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

7. Modern Data Solutions: End To End Flow
DATA SOURCES
INGEST
PREPARE
ANALYZE
PUBLISH
CONSUME
Diagnostic streaming
Event hub
Stream analytics
Power BI dashboards
Sensors and devices
Stream analytics
Machine learning
Cortana
HDInsight
HDInsight, machine learning
Business apps
Azure Data Lake
Azure Blob
Azure DW
Data Factory: move data, orchestrate, schedule, and monitor
Data Catalog: register, annotate, understand, discover data sets
Enterprise data sources

8. Modern Data Solutions: End To End Flow
DATA SOURCES
INGEST
PREPARE
ANALYZE
PUBLISH
CONSUME
Diagnostic streaming
Event hub
Stream analytics
Power BI dashboards 👆 👆 👆
Sensors and devices
Stream analytics
Machine learning 👆 👆 👆 👆 👆
Cortana 👆 👆
HDInsight
HDInsight, machine learning 👆
Business apps 👆
Azure Data Lake
Azure Blob
Azure DW 👆 👆
Data Factory: move data, orchestrate, schedule, and monitor
Data Catalog: register, annotate, understand, discover data sets
Data catalog: register, annotate, understand, discover data sets
Enterprise data sources

9. 11/7/2018
Solution Domain
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

10. What is Azure Data Catalog?
11/7/2018
What is Azure Data Catalog?
An enterprise-wide catalog in Azure that enables self- service discovery of data from any source
A metadata repository that allow users to register, annotate, discover, understand, and consume data sources
Slide goal: Introduce Azure Data Catalog before demo
A platform with open REST APIs that allow developers to integrate data discovery capabilities into their applications and processes
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11. Discover Understand Consume Contribute Search Browse Filter Metadata
11/7/2018
Manage & Curate
Search
Browse
Filter
Discover
Metadata
Experts
Context
Understand
Your data
Your tools
Your way
Consume
Tag
Document
Register
Contribute
Extend & Integrate
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

12. Common Customer Profile Patterns
11/7/2018
Common Customer Profile Patterns
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

13. Customer Maturity Profile
11/7/2018
Customer Maturity Profile
Mature enterprise companies recognize the need to treat data as an asset, and that getting more value from existing data sources is dependent on understanding the data and its function
These companies often have a Chief Data Officer or similar position with a senior leader from the business – not from IT – in charge of a systemic cultural change
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

14. What are customers looking for
11/7/2018
What are customers looking for
Each company has specific needs and motivations for initiating major strategic changes, but there are common trends and patterns:
Self-service business intelligence – companies need to enable more data consumers to analyze more data with less IT involvement
Self-service data preparation – companies need to make data available on demand for more purposes, without heavy IT investment
Big Data / Cloud Data and the “modern data warehouse” or data lake – companies recognize the long-term potential of data lakes to address problems and opportunities not yet identified
Cloud migration – companies want to move data sources, systems, and applications to the cloud, and need a deep understanding of existing assets to guide and support the migration
In addition to these themes, some customers have simply reached a point where the pain and cost inherent in the status quo is longer acceptable
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

15. Common Adoption Patterns
11/7/2018
Common Adoption Patterns
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

16. Bottom-Up Adoption Pattern Advantages Disadvantages
11/7/2018 7:41 PM
Bottom-Up Adoption
Pattern
Data Catalog provisioned by an individual department
Department users register, annotate, and consume
Loosely assigned responsibilities
Organic growth within the department and to neighboring departments
Advantages
Immediate business-driven value
Start small, evaluate, iterate, scale gradually
Disadvantages
No centralized strategy
No predictable growth
Inconsistent patterns of usage
© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

17. Top-Down Adoption Pattern Advantages Disadvantages
11/7/2018 7:41 PM
Top-Down Adoption
Pattern
Data Catalog adopted as part of larger data initiative
Population and ownership have well-defined responsibilities
Usage of data catalog incorporated into standard processes
Advantages
Centralized oversight and ownership
Standardized processes and points of contact ease adoption and collaboration
Easier to communicate and understand reach and ROI
Disadvantages
Value to existing “legacy” processes may be delayed or deprioritized
© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

18. Glossary-First Adoption
11/7/2018 7:41 PM
Glossary-First Adoption
Pattern
Focus is placed on populating the Business Glossary before populating the Data Catalog
Business taxonomies are fully and proactively defined
Registration of data assets includes annotation with business terms for assets and attributes
Advantages
Consistent and standardized quality and completeness of annotation and categorization for registered data sources
Optimized discovery experience for data consumers
Business glossary hierarchies serve as channels for discovery and exploration
Disadvantages
Time to value can be delayed
Effort involved in registration can be increased
© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

19. Variations on a Theme 11/7/2018
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

20. Ambient Catalog Population
11/7/2018 7:41 PM
Ambient Catalog Population
Pattern
Registering and updating catalog metadata is coordinated with data ingress processes
Data source metadata is provided via the Data Catalog REST API
Advantages
Data source metadata can be more consistent, current, and complete
Data Catalog integration can be built into – and maintained in conjunction with – data ingress processes for the data lake or data warehouse
Disadvantages
Lack of built-in integration points between Azure data services
Manual integration effort via REST API
© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

21. Cloud Migration Pattern Advantages Disadvantages
11/7/2018 7:41 PM
Cloud Migration
Pattern
Registering existing data sources and data assets to understand scope of starting point and guide prioritization and planning
Registering and annotating created data assets to improve ROI
Advantages
Core migration processes can be accelerated and optimized
Communicating progress and completion is easier
Data sources and assets can be mapped to business processes and terms
Disadvantages
No direct support for migration tools and workflows
© 2014 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

22. Q&A 11/7/2018 PGI call to action
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23. 11/7/2018
© 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.