BI Strategies For The Cloud Paresh Motiwala, PMP®

Setting up BI Infrastructure BI Curious Bi-Curious

Linkedin.com/in/pareshmotiwala Facebook.com/pareshmotiwala BI Strategies For The Cloud PareshMotiwala@gmail.com Linkedin.com/in/pareshmotiwala Facebook.com/pareshmotiwala Twitter: @pareshmotiwala 781 254 4096 @PASS_DBA_VC

BI Strategies For The Cloud

BI Strategies For The Cloud DBA Manager at Nuance Communications Leadership: Boston Business Intelligence User Group New England SQL Server User Group PASS DBA Virtual Chapter PASS Professional Development Virtual Chapter Organizer: Boston SQL Saturday Boston BI SQL Saturday Providence SQL Saturday Boston: Global Azure Bootcamp Principal: www.circlesofgrowth.com Speaker: SQL Saturdays, PASS, SQL Rally, UGs, Global Azure Boot Camps, Boston Code Camp Speaker Idol Finalist 2018 PASS Summit

BI Strategies For The Cloud Upcoming Events Boston BI SQL Saturday: March 30th

BI Strategies For The Cloud Who Should Attend? DBAs Developers DevOps Managers CIOs Infra folks Finance folks Cloud Enthusiasts

BI Strategies For The Cloud Points to consider Decide on workflow/applications to move Measure the current footprint Benchmark current performance Vendor selection Design the architecture(Azure DW or SnowFlake) SSRS SSIS Copy Data Management Security/GDPR Train the staff Reassess

BI Infrastructure – Sources

BI Strategies For The Cloud Applications to Move: Decide on workflow/applications to move Check with vendor Decide if you are overhauling it Lift and Shift Scope!

BI Strategies For The Cloud Footprint: Measure the footprint: CPU Memory Disk Space Type

BI Strategies For The Cloud Cloud Vendors

IaaS Vs PaaS Courtesy: Meagan Longoria and Melissa Coates

Concerns about the Cloud Technical: ✓ Uptime guarantees ✓ Performance ✓ Security ✓ Sharing of resources (multi-tenancy; noisy neighbors) ✓ Connecting legacy systems (hybrid/on-prem)

Concerns about the Cloud Management and Planning: ✓ Sprawl of self-provisioned services ✓ Compliance, regulations, legal ✓ Data and intellectual property privacy ✓ Vendor lock-in/dependency ✓ Lack of cloud expertise ✓ Complexity ✓ Cost and ongoing expenses ✓ Difficult to estimate cost up-front

BI Strategies For The Cloud Data platform continuum PaaS & SaaS Azure SQL Database Virtualized Database SQL Shared lower cost IaaS SQL Server in Azure VM Virtualized Machines Virtual SQL Server Private Cloud Virtualized Machine + Appliance Objective: One of the first things to understand in any discussion of Azure versus on-premises SQL Server databases is that you can use it all. The Microsoft data platform leverages SQL Server technology and makes it available across physical on-premises machines, private cloud environments, third-party hosted private cloud environments, and public cloud. This enables you to meet unique and diverse business needs through a combination of on-premises and cloud-hosted deployments, while using the same set of server products, development tools, and expertise across these environments. Talking Points: As seen in the diagram, each offering can be characterized by the level of administration you have over the infrastructure (on the X axis), and by the degree of cost efficiency achieved by database level consolidation and automation (on the Y axis). When designing an application, four basic options are available for hosting the SQL Server part of the application: SQL Server on nonvirtualized physical machines SQL Server in on-premises virtualized machines (private cloud) SQL Server in Azure Virtual Machine (public cloud) Azure SQL Database (public cloud) Physical SQL Server Physical Machine (raw iron) SQL Dedicated higher cost Higher administration Lower administration

BI Strategies For The Cloud: Objectives Easy Management All Data in One System Fast Elasticity Unlimited Concurrency Current technologies need constant attention, tuning, and tweaking for sub-optimal performance Current technologies are optimized for relational or non-relational data, leading to silos and data marts Ad-hoc queries are difficult to support with fixed resources Concurrency and contention lead to queues, long query times, and frustrated users © 2018 Snowflake Computing Inc. All Rights Reserved.

BI Strategies For The Cloud: 3 Roadblocks ’70s ‘80s ‘90s ‘00s ‘10s Scale Silos Concurrency

BI Strategies For The Cloud: Scaling??? Traditional databases are inflexible Usage varies, cost does not Adding capacity is time consuming Additional capacity is an expensive up-front purchase Enormous wasted capacity Usage varies

BI Strategies For The Cloud: Types of Data Well-Structured Semi-Structured { } Relational Database Blob Storage (S3, Azure Files) NoSQL Hadoop

BI Strategies For The Cloud : Modern Concurrency Data science ETL and Processing SQL analysts BI & analytics tools One resource for many groups of many people

BI Strategies For The Cloud: A Brand New Architecture Cloud Services Management Optimization Security Availability Transactions Metadata Data Science (L) Data Transformation (M) Structured Semi-Structured Structured Semi-Structured SQL Queries (S) BI/Reporting (Multi-Cluster M) Flexible Cloud Storage For All Kinds of Data Infinite Concurrency Unlimited Compute Clusters to Serve Every Use Easy-to-use Service with No Management © 2018 Snowflake Computing Inc. All Rights Reserved. Multi-Tenant Snowflake Deployment

BI Strategies For The Cloud: Azure DW Microsoft Large amounts of data You are already in Azure You need relational and non-relational data For large chunks of transactions Not for small chunks of data/OLTP Resides on Azure premium storage Always divided into 60 slices Gen 1 1006,000 DWUs Gen 2 1,000 30,000 DWUs DWU Max Concurrency, Memory Allocation and PolyBase readers/writers © 2018 Snowflake Computing Inc. All Rights Reserved.

BI Strategies For The Cloud BI Infrastructure – Reporting OLTP MDM/MDS Data Warehouse Reporting server Delivery Formats Dupe SSAS SharePoint Reporting server replica Data Lake Shareplexed BigData

BI Strategies For The Cloud BI Infrastructure – ETL OLTP SSMS BIDS/SSDT Export/Import Excel Access SSIS or SQL Server Tools Data Warehouse Dupe Shareplexed SSIS or SQL Server Data Lake BigData

BI Strategies For The CloudBI Infrastructure – At a glance MDM

BI Strategies For The Cloud Setting it up Garner Exec buy-in Define Measures LOFT Foundation Agile/Tools Constant Innovation Reassess

BI Strategies For The Cloud - Security GDPR Inform Involve Secure Train Audit Reassess

BI Strategies For The Cloud– Users Inform Involve Collaborate Train Equip Reassess

BI Strategies For The Cloud– Team Business Applications Database Project Mgt Senior Management Reassess

BI Strategies For The Cloud Source-Copy Data Definition Time Saving Secure Space Saving N/W Bandwidth Reassess

BI Strategies For The Cloud – Measuring Success

BI Strategies For The Cloud Modern Data Warehouse INGEST STORE PREP & TRAIN MODEL & SERVE Logs (unstructured) Azure Databricks Media (unstructured) Azure Data Factory PolyBase Files (unstructured) Azure Blob Storage Azure SQL Data Warehouse Azure Analysis Services Power BI Business/custom apps (structured) Microsoft Azure also supports other Big Data services like Azure HDInsight and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs.

BI Strategies For The Cloud Real time analytics STORE INGEST PREP & TRAIN MODEL & SERVE Sensors and IoT (unstructured) Logs (unstructured) Azure Databricks Apache Kafka for HDInsight Cosmos DB Real-time apps Media (unstructured) Microsoft Azure supports other services like Azure HDInsight, Azure Data Lake, Azure IoT Hub, Azure Events Hub in various layers of the architecture above to allow customers a truly customized solution. Files (unstructured) Azure Data Factory PolyBase Business/custom apps (structured) Azure Blob Storage Azure SQL Data Warehouse Azure Analysis Services Power BI Microsoft Azure also supports other Big Data services like Azure IoT Hub, Azure Event Hubs, Azure Machine Learning and Azure Data Lake to allow customers to tailor the above architecture to meet their unique needs.

BI Strategies For The Cloud Hub & Spoke architecture for bi INGEST STORE PREP & TRAIN MODEL & SERVE Business/custom apps (structured) Data Marts SQL Azure Databricks Multiple Azure SQL Database instances PolyBase Logs (unstructured) Azure Data Factory Azure Blob Storage Azure SQL Data Warehouse Power BI Media (unstructured) Data Cubes Multiple Azure Analysis Services instances Files (unstructured) Microsoft Azure supports other services like Azure HDInsight and Azure Data Lake in various layers to allow customers a truly customized solution.

BI Strategies For The Cloud Azure Databricks

BI Strategies For The Cloud Data Mart Consolidation INGEST STORE MODEL & SERVE PolyBase RDBMS data marts Azure Data Factory Azure Blob Storage Azure SQL Data Warehouse Azure Analysis Services Power BI Hadoop Microsoft Azure also supports other Big Data services like Azure HDInsight and Azure Data Lake to allow customers to tailor the architecture to meet their unique needs.

BI Strategies For The Cloud CHOOSING THE DATABASE Managed Instance Elastic databases Single databases Cloud-born SaaS applications ISV  SaaS applications Custom customer-facing applications Employee-facing applications Applies to New SaaS providers Existing SQL ISV re-architecting for SaaS All entities Enterprises building custom apps Goal Compete and replace on-premises competitor apps Replace existing LOB packaged software with a SaaS solution Scale as the customer base grows Create or re-architect apps to cloud or use SaaS solutions Needs Hyper-scale, HADR, performance Write code once Predictable and flexible cost SQL compatibility Time to solution Packaged LOB apps – delivered by ISVs Objective: Microsoft worked closely with some customers during their initial onboarding to Azure SQL Database to learn how they used the service and take lessons back to our engineering team for future feature planning. Talking Points: During those engagements, we found that some kinds of customers found the feature set suited their needs well. For example, startups developing new cloud services often found that the combination of capacity on demand and reduced administrative overhead simplified their lives and allowed them to focus time on their core business. Other customers had challenges in some areas related to tight performance requirements―perhaps for service a central API in a large, multi-tier database solution―that were not currently met by the Azure SQL Database service. The feedback was that while some customers were very willing to accept higher performance variance to achieve a very low price point, other customers were more interested in specific performance guarantees so they could more easily build higher-level value on top of these databases.

BI Strategies For The Cloud Recommendation Recommend one or more of the strategies. Summarize the results if things go as proposed. What to do next. Identify action items.

BI Strategies For The Cloud Points to consider Decide on workflow/applications to move Measure the current footprint Benchmark current performance Vendor selection Design the architecture SSRS SSIS Copy Data Management Security/GDPR Train the staff Reassess

Linkedin.com/in/pareshmotiwala Facebook.com/pareshmotiwala BI Strategies For The Cloud Contact Information PareshMotiwala@gmail.com Linkedin.com/in/pareshmotiwala Facebook.com/pareshmotiwala Twitter: @pareshmotiwala 781 254 4096 @PASS_DBA_VC

BI Strategies For The Cloud Thank You BI Strategies For The Cloud Remember Paresh For President 2020 Thank You

BI Strategies For The Cloud BI Infrastructure – Bibliography Howsen, C. (n.d.). Successful Business Intelligence, Unlock the Value of BI & Big Data. Kolb, J. (n.d.). Business intelligence in plain language: A practical guide to data mining and business analytics. Sharda, R., & Delen, D. (n.d.). Business Intelligence: A Managerial Perspective. White Paper- Technovision 2014 , Digital Transformation by Capgemini Microsoft’s explanation of Hadoop http://bit.ly/1HylMT6 BI news from Microsoft http://bit.ly/1E0S19B Cloud Workloads presentation from Jose Valencia, Microsoft Elastic Data Warehouse as a Service - Snowflake