Managing Data in the Cloud 📈 ☁ @JeremyLikness Microsoft

Jeremy Likness https://blog.jeremylikness.com/ @JeremyLikness Cloud Advocate @ Microsoft Quarter Century Professional Developer 👟 Likes to “run” 🥑 99.9765% plant-based diet 🎱 Shoots pool whenever possible   https://blog.jeremylikness.com/ @JeremyLikness Jeremy.Likness@microsoft.com @JeremyLikness

Three Examples @JeremyLikness

Airports and flight paths Flights Database Airports and flight paths @JeremyLikness

@JeremyLikness

@JeremyLikness

@JeremyLikness

@JeremyLikness

Link Tracking “A Tale of Short Links” @JeremyLikness

@JeremyLikness

@JeremyLikness

@JeremyLikness

@JeremyLikness

Tailwind Traders Inventory Microservices and Front Ends @JeremyLikness

@JeremyLikness

@JeremyLikness

Volume, Variety, and Velocity Classify Your Data Volume, Variety, and Velocity @JeremyLikness

Taxonomical Structured Semi-Structured Unstructured Fixed schema Relational Semi-Structured Properties with values Tags Hierarchy Unstructured Streams Metadata present Media, logs, text @JeremyLikness

Operational Transactions required? Size of data set? Volume of inserts/updates? Geographic distribution Local laws (geo-fencing) Resiliency @JeremyLikness

Solutions Structured SQL MySQL PostreSQL Semi-Structured Key/Value (Azure Tables) Graph (Gremlin) Document (MongoDB, DocumentDB) Unstructured Azure Files Azure Blob Storage @JeremyLikness

Unstructured Data Files and Blobs @JeremyLikness

Accounts Azure SQL Azure Cosmos DB Azure Storage 1 … N databases 1 API 1 … N “containers” (collection, graph, or table) Azure Storage 1 … N blob containers 1 … N file shares 1 … N table storage 1 … N queues Accounts

Massively Scalable Cloud Storage Durable and available Secure Scalable Managed Accessible

Accounts: A Unit of Scale and Deployment General purpose v2 Latest features Lowest per-gigabyte cost More options for resiliency General purpose v1 Limited in features May not be lowest cost Can upgrade “in place” to v2 Blob storage Specialized if you are only leveraging the blob storage features of an account

Your Azure Storage Friends Storage Emulator Part of the Azure SDK Also a standalone installer https://jlik.me/fsl Storage Explorer Cross-platform (Linux, macOS, Windows) Azure and local storage https://jlik.me/fsm Your Azure Storage Friends

Demo: Create a Storage Account

Data Redundancy Geo Redundant (GRS) Zone Redundant (ZRS) Secondary region hundreds of miles away High availability Highest cost Zone Redundant (ZRS) 12 9’s 3 storage clusters in a region Zone can be unavailable and still will work Not safe if entire region goes down Locally Redundant (LRS) 11 9’s Spread across local replicas Lowest cost but least durability (loss of DC means loss of data)

Blob Storage Images Documents Static websites Video and audio Log files Disaster recovery Data Analysis

Blob “Flavors” Block Append Page Text or binary up to 4.7 TB Optimized for appending (i.e. logs) Page Paged on disk for Virtual Hard Disks Blob “Flavors”

Demo: Blob Storage

Access Tiers Premium Hot Cool Archive Solid State Drives High storage cost Low access cost Fast Cool Low storage cost High access cost Archive Lowest storage cost Highest access cost 15 hours to access Access Tiers

Blob Features Encryption Shared Access Signature (SAS) Role-Based Access Control (RBAC) Access Policies Events Static Websites

Demo: Write to Blob Storage @JeremyLikness

Azure Files @JeremyLikness

Azure Files Server Message Block (SMB) Windows, Linux, and macOS Replication with Azure File Sync Support “Lift and Shift” Common Mounts (Tools, Images, etc.) Azure Files

Azure Files Benefits Shared Access Fully Managed Script and Tool Friendly Resilient “Always Available” Program to Standard I/O Azure Files Benefits

Semi-Structured Data NoSQL @JeremyLikness

8/8/2019 10:29 PM @JeremyLikness © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Disconnect: Relational vs. Domain 8/8/2019 10:29 PM Disconnect: Relational vs. Domain Relational Database ORM Object Object @JeremyLikness © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

8/8/2019 10:29 PM Have you ever … Created a table named metadata and populated it with columns that have names like: Column_name Column_type Column_string_value Column_integer_value Stuffed XML, JSON, or other miscellaneous data into a large nvarchar(max)? Better yet, used the XML/JSON field types and tried to write a query over them? OUCH! @JeremyLikness © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Introducing Not Only SQL: NoSQL 8/8/2019 10:29 PM Introducing Not Only SQL: NoSQL Key/Value Column Document Graph 🗝 🗝 🏛 🏛 📄 📄 📈 📈 @JeremyLikness © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

🗝 🗝 Key/Value @JeremyLikness 8/8/2019 10:29 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

🏛 🏛 Column @JeremyLikness 8/8/2019 10:29 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

📈 📈 Graph @JeremyLikness 8/8/2019 10:29 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

📄 📄 Document @JeremyLikness 8/8/2019 10:29 PM © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Demo: NoSQL Link, metadata and flight information

Serverless Link Shortener ShortenURL Insert Query Read Enqueue Redirect UpdateTwitter Read Dequeue Insert Update @JeremyLikness

Structured Query Language, anyone? Structured Data Structured Query Language, anyone? @JeremyLikness

@JeremyLikness

Demo: Azure SQL @JeremyLikness

@JeremyLikness

Getting your Data There Data Migration Services @JeremyLikness

DEMO: Data Migration Services https://jlik.me/fsy @JeremyLikness

Concluding Thoughts Relational data isn’t the only solution Classify your data Hybrid (mixed) solutions are fine @JeremyLikness

💻 Storage Emulator: https://jlik.me/fsl 8/8/2019 10:29 PM https://blog.jeremylikness.com/ @JeremyLikness Jeremy.Likness@microsoft.com 💻 Storage Emulator: https://jlik.me/fsl 🔍 Storage Explorer: https://jlik.me/fsm 💾 Azure Storage: https://jlik.me/fsz 🌍 Azure Cosmos DB: https://jlik.me/fs0 📦 Azure SQL Database: https://jlik.me/fs1 👨‍💻 Source Code/Demo: https://jlik.me/fs3 © Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.