Download presentation
Presentation is loading. Please wait.
1
Managing Data in the Cloud π β
@JeremyLikness Microsoft
2
Jeremy Likness https://blog.jeremylikness.com/ @JeremyLikness
Cloud Microsoft Quarter Century Professional Developer π Likes to βrunβ π₯ % plant-based diet π± Shoots pool whenever possible @JeremyLikness @JeremyLikness
3
Three Examples @JeremyLikness
4
Airports and flight paths
Flights Database Airports and flight paths @JeremyLikness
5
@JeremyLikness
6
@JeremyLikness
7
@JeremyLikness
8
@JeremyLikness
9
Link Tracking βA Tale of Short Linksβ @JeremyLikness
10
@JeremyLikness
11
@JeremyLikness
12
@JeremyLikness
13
@JeremyLikness
14
Tailwind Traders Inventory
Microservices and Front Ends @JeremyLikness
15
@JeremyLikness
16
@JeremyLikness
17
Volume, Variety, and Velocity
Classify Your Data Volume, Variety, and Velocity @JeremyLikness
18
Taxonomical Structured Semi-Structured Unstructured Fixed schema
Relational Semi-Structured Properties with values Tags Hierarchy Unstructured Streams Metadata present Media, logs, text @JeremyLikness
19
Operational Transactions required? Size of data set?
Volume of inserts/updates? Geographic distribution Local laws (geo-fencing) Resiliency @JeremyLikness
20
Solutions Structured SQL MySQL PostreSQL Semi-Structured
Key/Value (Azure Tables) Graph (Gremlin) Document (MongoDB, DocumentDB) Unstructured Azure Files Azure Blob Storage @JeremyLikness
21
Unstructured Data Files and Blobs @JeremyLikness
22
Accounts Azure SQL Azure Cosmos DB Azure Storage 1 β¦ N databases 1 API
1 β¦ N βcontainersβ (collection, graph, or table) Azure Storage 1 β¦ N blob containers 1 β¦ N file shares 1 β¦ N table storage 1 β¦ N queues Accounts
23
Massively Scalable Cloud Storage
Durable and available Secure Scalable Managed Accessible
24
Accounts: A Unit of Scale and Deployment
General purpose v2 Latest features Lowest per-gigabyte cost More options for resiliency General purpose v1 Limited in features May not be lowest cost Can upgrade βin placeβ to v2 Blob storage Specialized if you are only leveraging the blob storage features of an account
25
Your Azure Storage Friends
Storage Emulator Part of the Azure SDK Also a standalone installer Storage Explorer Cross-platform (Linux, macOS, Windows) Azure and local storage Your Azure Storage Friends
26
Demo: Create a Storage Account
27
Data Redundancy Geo Redundant (GRS) Zone Redundant (ZRS)
Secondary region hundreds of miles away High availability Highest cost Zone Redundant (ZRS) 12 9βs 3 storage clusters in a region Zone can be unavailable and still will work Not safe if entire region goes down Locally Redundant (LRS) 11 9βs Spread across local replicas Lowest cost but least durability (loss of DC means loss of data)
28
Blob Storage Images Documents Static websites Video and audio
Log files Disaster recovery Data Analysis
29
Blob βFlavorsβ Block Append Page Text or binary up to 4.7 TB
Optimized for appending (i.e. logs) Page Paged on disk for Virtual Hard Disks Blob βFlavorsβ
30
Demo: Blob Storage
31
Access Tiers Premium Hot Cool Archive Solid State Drives
High storage cost Low access cost Fast Cool Low storage cost High access cost Archive Lowest storage cost Highest access cost 15 hours to access Access Tiers
32
Blob Features Encryption Shared Access Signature (SAS)
Role-Based Access Control (RBAC) Access Policies Events Static Websites
33
Demo: Write to Blob Storage
@JeremyLikness
34
Azure Files @JeremyLikness
35
Azure Files Server Message Block (SMB) Windows, Linux, and macOS
Replication with Azure File Sync Support βLift and Shiftβ Common Mounts (Tools, Images, etc.) Azure Files
36
Azure Files Benefits Shared Access Fully Managed
Script and Tool Friendly Resilient βAlways Availableβ Program to Standard I/O Azure Files Benefits
37
Semi-Structured Data NoSQL @JeremyLikness
38
8/8/ :29 PM @JeremyLikness Β© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
39
Disconnect: Relational vs. Domain
8/8/ :29 PM Disconnect: Relational vs. Domain Relational Database ORM Object Object @JeremyLikness Β© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
40
8/8/ :29 PM Have you ever β¦ Created a table named metadata and populated it with columns that have names like: Column_name Column_type Column_string_value Column_integer_value Stuffed XML, JSON, or other miscellaneous data into a large nvarchar(max)? Better yet, used the XML/JSON field types and tried to write a query over them? OUCH! @JeremyLikness Β© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
41
Introducing Not Only SQL: NoSQL
8/8/ :29 PM Introducing Not Only SQL: NoSQL Key/Value Column Document Graph π π π π π π π π @JeremyLikness Β© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
42
π π Key/Value @JeremyLikness 8/8/2019 10:29 PM
Β© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
43
π π Column @JeremyLikness 8/8/2019 10:29 PM
Β© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
44
π π Graph @JeremyLikness 8/8/2019 10:29 PM
Β© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
45
π π Document @JeremyLikness 8/8/2019 10:29 PM
Β© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
46
Demo: NoSQL Link, metadata and flight information
47
Serverless Link Shortener
ShortenURL Insert Query Read Enqueue Redirect UpdateTwitter Read Dequeue Insert Update @JeremyLikness
48
Structured Query Language, anyone?
Structured Data Structured Query Language, anyone? @JeremyLikness
49
@JeremyLikness
50
Demo: Azure SQL @JeremyLikness
51
@JeremyLikness
52
Getting your Data There
Data Migration Services @JeremyLikness
53
DEMO: Data Migration Services
@JeremyLikness
54
Concluding Thoughts Relational data isnβt the only solution
Classify your data Hybrid (mixed) solutions are fine @JeremyLikness
55
π» Storage Emulator: https://jlik.me/fsl
8/8/ :29 PM @JeremyLikness π» Storage Emulator: π Storage Explorer: πΎ Azure Storage: π Azure Cosmos DB: π¦ Azure SQL Database: π¨βπ» Source Code/Demo: Β© Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Similar presentations
