Change Is Easy! Managing Change is Hard James McAuliffe BI Solution Architect, MCITP CCG Analytics Change Is Easy! Managing Change is Hard A Walk Around SQL Server Tools for Databases

Agenda – Walk Arounds Database Change Management Principals Why Is it Important Overview of Database Development Lifecycle Change Management Approaches Concepts for Development And Deployment Manual Change Management Options Introduce SQL Server Data Tools for Databases (VS2017) Walkaround the Tool – How Does it Fit? Demos – About 30-35% Intermediate for concepts, less complicated for tooling Database Change Management Principals Why Is it Important Overview of Database Development Lifecycle Change Management Approaches Concepts for Development And Deployment Manual Change Management Options Introduce SQL Server Data Tools for Databases (VS2017) Walkaround the Tool – How Does it Fit? Demos – About 30-35% Intermediate for concepts, less complicated for tooling

CCG Analytics Solutions & Services CCG is an award winning consulting firm that values our Culture, our Community and our continued Growth “Work hard, play hard” Culture Chartered Community Council Award winning: Sustained Growth of 20-30% year Best Places to Work Fast 50, Inc 5000

About Me BI Solution Architect, Cloud Solution Architect, Data Platform Tampa Native USF Alumni: Electronic Music Composition Travelled Country Doing High School Assemblies Built commercial cleaning business in Florida, Georgia, Texas, South Carolina. Developed software on which the business ran Entered IT/MIS in the 90s and loving it ever since – first gig was with Pam Shaw 4 Daughters Certs in Database Development, Business Intelligence, Microsoft Solution Framework, etc.

CCG Clients

My Spare Time Automobile Functional Decomposition Chaos to Order

About You: Why Are You Here? Roles? Industry: What Types of Businesses? What Do You want to get out of this? About You Roles Technologists Business CIO - CTO - CDO MIS Recent Graduates Architects Industries: Financial Retail Manufacturing Experience in Azure?? Are you a: Database developer? Database Administrator? BI Developer? Other --?? I consider myself a database developer, but a lot of times I have to function in the DBA role. This session is an entry level session for: Database Developers – The people who make the change Database Administrators – The people who deliver the change PEOPLE WHO DEPEND ON DATA FOR A LIVING Client with 5 database environments Develop/Sandbox Collaborative Development Test UAT (off premise) Production We were discussing the deployment strategy, and how this tool was going to be used, and the handoff to the DBA One of the db developers said: “This tool seems really cool.” So I added more discussion on the WHY….. Presentation will be balanced between talking about the change and product demo For the Database Developers Want to MAKE change Don’t want to have to care about the change Maybe YOU have to deliver the scripts….. Let’s make that easier For the Database Administrators Want to CONTROL change Don’t want to lose their data Want everything to work Want to be able to capture all the changes NEED to be able to introduce change without destroying the database

Why Do You Care? Volume and Pressure 44 ZB (updated) 5 exabytes is ALL words ever spoken by the entire human race, through history. more than 5,200 gigabytes for every man, woman, and child in 2020 By 2020, data will reach 44ZB, quadruple what it is today Measuring Impact Terabyte: 1000 Gigabytes Petabyte: 1000 Terabytes Exabyte : 1000 Petabytes   You do not see curves like this very often: think of it as surfing a wave: if you don’t catch it at the beginning, you can never catch up. So this type of thing represents either the BIGGEST THREAT or AMAZING OPPORTUNITY HOW does this impact your company's data strategy? Think about your current company’s data strategy? Let’s look at what this new data looks like

Change Is Going to Happen: Inevitable Easy To Introduce Hard to Manage Expensive Sometimes TOO expensive Change is Easy – Managing Change Is Hard Change Is Inevitable - if your company is growing – that drives change - if your company is shrinking – that drives change 2. Easy to Introduce - in high demand - database developers “Just this one field”, “just this one…. Whatever” 3. Hard to Manage - We don’t often see the value of saying NO - no one know about the downstream effects - hard to have visibility into the cumulative effects of lots of little changes made by lots of people. 4. Expensive -making change is expensive … but not TOO Expensive:

Data Is Valuable Your Cell Phone Data Your Digital Pictures Your Company’s Data Marketing Contacts Operations Payroll Financial Reporting What are some data you have that is valuable? Cell phone, Pictures on your computer.

Data Is Expensive Expensive to Acquire Expensive to Retain Expensive to LOSE

Modern Data Warehouse: Deliver and Enable INGEST STORE PREP & TRAIN MODEL & SERVE Logs (unstructured) Azure Data Factory Move and Transform Azure SQL Database Media (unstructured) Azure Data Factory Azure Databricks Files (unstructured) Azure Blob Storage Azure Data Lake Store Gen 2 Azure Analysis Services Power BI Azure SQL Database This type of Database Development Lifecycle is centered around the presentation layer, but the governance of the data management goes up and down the data management and lineage chain. Azure data factory to ingest from any source and transform data into any required shape Assume ADF pipelines Assume SSIS (Integration Runtime) Assume Databricks capabilities (Python, Spark, etc) Unstructered and file-based data is landed in the DataLake and runs through any process needed using the tooling indicated. Structured data can go directly to a relational repository that will contain Staging -> Integration (ODS/Inmon rationalization area) Data from the DataLake or the relational repository can be moved into the reporting dimensional layer using a combination of ADF, SSIS, or databricks, depending on the source of the data required. The Azure SQL Database in the “Model and Serve” section is a dimensional repository that is a final data presentation layer. Azure Analysis services is used for modeling capabilities and to provide a universal semantic layer. PowerBI Is used for the visualization reporting Note that PBI can also blend and present Iot Streaming, uncurated, or data from other sources Analytic Enablement Azure SQL Database Business/custom apps (structured) Microsoft Azure also supports other Big Data services like Azure HDInsight and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs.

Database Development Lifecycle (Relational) Analysis Logical Design Physical Develop Deliver Maintain A DDLC, by process, is not that different from a SDLC. Data CONTENT, and the value of the data, makes the delivery and maintenance of the data entities more challenging. Envision Plan/Design Build/Develop Deploy (with test!) Maintain/Stabilize

Data Change Delivery Is Challenging class AuctionApplication ( int id; void MethodA(); ) class AuctionApplication ( int id; void MethodA(); void MethodB(); ) class AuctionApplication ( int id; string cacheTitle; void MethodA(); void MethodB(); ) App V 1 Revision History V 2 V 3 CREATE TABLE dbo.Auction ( id INT NOT NULL, name VARCHAR(25) NOT NULL, start DATETIME NULL, len INT NULL ) ALTER TABLE dbo.Auction WITH CHECK ADD CONSTRAINT Au_PK PRIMARY KEY (id) ALTER TABLE dbo.Auction WITH CHECK ADD CONSTRAINT Au_SK UNIQUE (name) Database This slide originally created by Gert Drapers for Team Database Developer GDR2. So, with DDLC, our versioning challenge is quite different than SDLC. So let’s talk about the versioning challenge: Take an example with of a revision chronology or history…. Compare DEV development to Database Development. Distinct roles, similar functionality: responsible for change. App Dev rolls on (simple example – we are not talking about data bound UI stuff here – don’t get confused) Database development beyond Difference is that database development is STATEFUL…. We have to be aware of what is there, and “change” it We have to preserve the data content. What does that currently look like?

T-SQL Change Management DEMO Create a table named AUCTION Populate the table. Make some changes. Figure out some stuff See database script project: T-SQL Scripts   Demo creating the AUCTION table. Delete existing table if there Create the table Populate the table with demo script 1 Make a stored procedure from that. Make the change(s) - talk about impacts Do typical things, allow mistakes to happen. You should notice problems with any dependent constraint or object. Repopulate the table – this should be broken. Discuss what happened and how we might mitigate the problems.

Review: Change Management Is Hard Change WILL Happen Data Is Valuable Data Is Expensive Data Loss is Even More Expensive Data Change Delivery Can Be Challenging Next: Make It Easy

Change Management Approach TechReady4 Change Management Approach Single Source Of Truth Organizational Support People and Processes to determine change (Governance) Structural Organization in the Library Development Support Deployment Compare Delta Scripts Maintenance Delta A good change management strategy will have AT LEAST the listed components ONE SOURCE OF TRUTH Organizational Support A single authoritative definition that everyone agrees is the standard. DEVELOPMENT Support Structural organization – find things (Sharepoint comparison) What, why, when, how MAINTENANCE Generate delta scripts Compare versions Gen Delta Scripts – introduce the change Compare © 2007 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Change Management is Hard – Make it Easy SQL Server Data Tools (SSDT) Free since SQL Server 2012 Free with Visual Studio Community Editions and up

One Source Of Truth Multiple Database Targets Multiple Developers Multiple Versions One Ring To Rule Them All - SSDT

Organize Having Lots of Stuff is Useless SSDT Uses Same Organization Model As SSMS Solution: Organize By Object Schema Schema/Object SQL Server Object Explorer (SSDT) = Object Explorer (SSMS) If it’s all the same to you… Organize!! (Just) having lots of stuff is useless – Sharepoint Anyone? How about a familiar interface?

SSMS vs SSDT – which is which? Both SSMS and SQL Server Object Explorer in SSDT use the same IDE design paradigm Show AW_Demo vs. AdventureWorks

Develop VS Developers and Database Developers Share Common Construct Developers can work either Offline Online When Done, Check Changes Into the DB Project Multiple Sources, One Destination

Deploy Handoff is continual When time for a release, all roles can see all changes consistently Database Developer OR Application Developer OR DBA Generate one script Life is good.

Visual Studio Interface Here is where you can find the tools. If you do not see SQL Server, then check the “Get Tools and Features”

DEMOS Open Visual Studio and Explore SQL Server Data Tools Open Existing Project without Source Control Capture from Database Make Changes w/o using Refactor Rename Show Snapshot management of Changes Revert to baseline and carry through Add Database Management to Source Control Open ChangeDemoAuction project - Create a table Show artifacts Tables Stored proc Refactor rename auction the change(s) - talk about impacts Repopulate the table.

Review: Change Management Approach TechReady4 Review: Change Management Approach Single Source Of Truth Organizational Support People and Processes to determine change (Governance) Structural Organization in the Library (Consistency) Development Support Deployment Compare Delta Scripts Maintenance Delta A good change management strategy will have AT LEAST the listed components ONE SOURCE OF TRUTH Organizational Support A single place that everyone agrees is the standard. DEVELOPMENT Support Structural organization – find things (Sharepoint) What, why, when, how MAINTENANCE Generate delta scripts Compare versions Gen Delta Scripts – introduce the change Compare © 2007 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

Resources SQL Server Data Tools https://docs.microsoft.com/en-us/sql/ssdt/sql-server-data-tools?view=sql-server-2017 SQL Tools and Utilities for SQL Server, Azure SQL Database, and Azure SQL Data Warehouse https://docs.microsoft.com/en-us/sql/tools/overview-sql-tools?view=sql-server-2017 Data Developer Center -> Microsoft SQL Server Data Tools https://msdn.microsoft.com/en-us/data/tools.aspx Learn More About Microsoft SQL Server Data Tools http://msdn.microsoft.com/en-us/data/hh297029 SQL Server Data Tools Team Blog http://blogs.msdn.com/b/ssdt/ Download and install SQL Server Data Tools (SSDT) for Visual Studio https://docs.microsoft.com/en-us/sql/ssdt/download-sql-server-data-tools-ssdt?view=sql-server-2017

Thank You!! Change Is Easy! Managing Change is Hard James McAuliffe CCG Analytics jmcauliffe@ccganalytics.com https://ccganalytics.com/ https://www.linkedin.com/in/jamesmcauliffesql/ Evaluations!!