Download presentation
Presentation is loading. Please wait.
1
Azure SQL Data Warehouse for Beginners
David Alzamendi Azure SQL Data Warehouse for Beginners
2
Thanks to our Sponsors
3
Lunch Time Session from Sponsors
4
Raffle Prize Draw: Don’t miss the opportunity
Enter Raffle Prize Draw for each sponsor Be at lecture Keynote room at 5pm $350 Flight Centre voucher Super Fast Dash - Hamptown Downs (worth $170) Two Vouchers for Power BI Training Videos (each worth $260 USD )
5
About Me
6
About Me I have been working as a Business Intelligence consultant the past few years. I have worked in all different areas of Business Intelligence solutions, providing high standard responses. Likewise, I have acquired international experience through living and working abroad in Argentina, Spain and Australia. Hobbies: Travel Cooking Sports @david_alzamendi
7
Get Involved “Sharing is Caring”
8
Today’s Session Introduction to ADW Architecture (Change of rules)
Demo 1 (Provisioning) Pros Cons Demo 2 (Polybase) Best Practices Security
9
Introduction to Azure Data Warehouse
Azure SQL Data Warehouse is a massively parallel processing (MPP) cloud-based, scale-out, relational database capable of processing large amounts of data.
10
Azure Data Warehouse
11
TIMELINE On-Premises Cloud 2008 2010 2011/2013 2014 2015
Microsoft Acquired DATAllegro, company that makes data warehouse appliances 2010 Microsoft Launched Fast Track Architecture Data Warehouse reference using SMP. DW best practices offered with leading H/W Partners. 2011/2013 Parallel Data Warehouse v1 and v2. Data Allegro product on Windows & SQL First DW appliance by Microsoft in partnership with Dell and HP 2014 Analytics Platform System (APS). Introduction of Hadoop region within appliance and new naming to reflect broader Big Data capabilities. Cloud 2015 Introduction of Azure SQL DW. Service based on APS’s MPP capabilities.
12
When to use Azure Data Warehouse
You plan to move your DW to the cloud Tight deadlines for delivering a DW solution When you don’t want to invest millions of dollars Not enough skills for infrastructure, security, network, etc..PAAS! Store large amounts of data Integrate data from one or many sources in a single database Run complex queries or Ad-hoc reports against large amounts of data
13
When NOT to use Azure Data Warehouse
OLTP Systems Row by row processing High number of single transactions Frequent reads and writes
14
VS Symmetric Multiprocessor System (SMP)
Architecture Massively Parallel Processing (MPP) VS
15
Architecture Tables Data is stored in 60 distributions
Round Robin: Distributes information randomly. Hash Distributed: Distributes data bases on hashing values from a single column Columnstore Index by default (10X compression + 100X query performance)
16
Load Data You can use load data using: Polybase Bcp Azure Data Factory
SSIS
17
But what is the difference?
Blob Storage Table Storage File Storage Queue Cosmos DB Azure DB Azure Data Lakes Azure DW
18
Azure Data Warehouse vs Azure Data Lakes
Structured Data Structured, semi structured and non-structured data Processing It has shape (schema-on-write) No, shape (schema-on-read) Agility Less agile, fixed configuration Highly agile, configure and reconfigure as needed Security Mature Maturing Final Users Business professionals All users (data scientists)
19
Azure Data Warehouse vs Azure SQL Database
Architecture Relational DB using MPP Relational DB using SMP System Online Analytical Processing (OLAP) Online Transaction Processing (OLTP) Storage Unlimited Up to 4 TB Polybase Yes No Concurrent Queries Up to 32 (not DMVs) Up to
20
Example
21
Provisioning ADW using: Azure Resource Manager Power Shell T-SQL
Demo 1 Provisioning ADW using: Azure Resource Manager Power Shell T-SQL
22
Azure Data Warehouse PROS
PaaS Azure Ecosystem Decouples storage from compute Scale Out Price Pause service Availability and Backups
23
PaaS
24
Azure Ecosystem More
25
Decouples storage from compute
Grow or shrink storage size independent of compute. Grow or shrink compute power without moving data. Pause compute capacity while leaving data intact, only paying for storage.
26
Scale Out Deploy it in a few minutes
From DW100 to DW6000 (DW9000, DW18000 * Public Preview) DW100 = 1 compute node DW6000 = 60 compute nodes DW 18000= 180 compute nodes Automatically Scale out using: T-SQL Rest API Power Shell
27
Price DWU (Data Warehouse Units)
(DWUs are a measure of underlying resources like CPU, memory, and I/O bandwidth which are allocated to your SQL Data Warehouse. Increasing the number of DWUs increases resources and performance.) DWU Calculator to rescue! 1 compute node (100 DWU) x $1.21/hour (USD) 1TB x $0.17/hour = $122/month (USD) 200$ free credit in your first Azure Subscription 25$/12 months Visual Studio Dev Essentials
28
Pause the Service Compute and memory resources are returned to the pool of available resources in the data center DWU costs are zero for the duration of the pause. Data storage is not affected and your data stays intact. SQL Data Warehouse cancels all running or queued operations. Pause and Resume using Power Shell, ARM and REST APIs.
29
Availability and Backups
Available in 30 regions (SE AUS Melbourne- SE ASIE Singapore) Locally Redundant Storage (LRD) for free Available Read Access Geo Redundant Storage (RA-GRS) Snapshot every 4 to 8 hours Retention for 7 days
30
Azure Data Warehouse CONS
Maintain statistics Data Types (geography,geometry,hierarchyid,image,text,ntext,sql_variant,timestamp,xml) Primary key, Foreign keys, Unique and Check Table Constraints Unique Indexes, Computed columns, User-Defined Types, Global temporary tables, Synonyms, Indexed Views, Triggers, Sequences In Memory OLTP is not available
31
Load data using Polybase: Create Credentials
Polybase rows /DW1000 Demo 2 Load data using Polybase: Create Credentials Create External Data Source Load Data using CTAS
32
Best Practices Pause the service Scale when necessary
Use SSAS on top of the ADW to increase concurrency (better together) Maintain Statistics Load the data using Polybase Hash large tables Not Over Partition Temporary Heap Tables to transform data Use Large Resource Class for Memory Consuming Transactions Use DMVS to monitor queries
33
Security TDE (Transparent Data Encryption) It doesn’t support:
Always Encrypted. Grant permission to different users through Schemas and Roles
34
Wrap Up Easy to use Go and create your first DW in a few minutes
PaaS DW Scale out and down as needed Low cost DW
35
Questions?
36
Thanks to our Sponsors
37
Don’t forget your feedback please!
Thank You! Don’t forget your feedback please! Documentation
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.