1. Implementing an Azure SQL Data Warehouse
Module 5
The 20767A-MIA-SQL virtual machine used in the lab for this module includes software services that can take a while to start. For the best experience, have students start the 20767A-MIA-DC and 20767A-MIA- SQL virtual machines at the beginning of the module so that the services are ready before they start the lab.
There are demonstrations and labs in this course which require access to Microsoft® Azure™. As such you need to allow sufficient time for the setup and configuration of a Microsoft Azure pass which will allow access to Microsoft Azure for you and your students.
Details of how to acquire Microsoft Azure passes for your class are available here:
Access to Microsoft Learning Azure Passes for Students of Authorized Microsoft Learning Partners
Implementing an Azure SQL Data Warehouse

2. Copying Data with the Azure Data Factory
Module Overview
5: Implementing an Azure SQL Data Warehouse
Copying Data with the Azure Data Factory

3. Lesson 1: Advantages of Azure SQL Data Warehouse
5: Implementing an Azure SQL Data Warehouse
Hybrid Cloud
Question
You are working on a data warehouse that sometimes requires executions with very high workloads, but also goes for long periods of time with a very low workload requirement. If you migrate this data warehouse to Azure SQL Data Warehouse, how can you improve the efficiency of resource allocation?
Answer
You can set the DWU performance to a high level during the periods when the workload requirements are high, and set the DWU performance to a low level during the periods when the workload requirements are low.

4. What is Azure SQL Data Warehouse?
5: Implementing an Azure SQL Data Warehouse
Cloud-Based Database
Relational and non-relational
Enterprise workloads
Integrated with Azure
Fully managed service
Benefits Include:
Massive parallel processing
Advanced query optimization
Columnstore indexes
PolyBase integration
Auditing
Scalability

5. Scalability and Cost No upfront cost Storage Compute
5: Implementing an Azure SQL Data Warehouse
No upfront cost
Storage
Adjusts automatically
Cost based on storage used
Compute
Determines execution performance
Data Warehouse Unit (DWU)
Increase or decrease DWU
Cost based on DWU used
Pause and start

6. Security and Availability
5: Implementing an Azure SQL Data Warehouse
Security
Firewall
Add logins
Set authorisation
Auditing
Availability
Can restore in different region
Can choose restore point in last seven days

7. Can access unstructured data in other systems
PolyBase
5: Implementing an Azure SQL Data Warehouse
Can access unstructured data in other systems
Set up external table to link to data source
Query external table as normal table
Mention the forward link in the manual for more information about setting up an external table in PolyBase.

8. Use PolyBase to query and copy data with Transact-SQL
Hybrid Cloud
5: Implementing an Azure SQL Data Warehouse
Can integrate between on-premises, cloud, and unstructured data sources
Use PolyBase to query and copy data with Transact-SQL
Schedule data copy using Azure Data Factory

9. Lesson 2: Implementing an Azure SQL Data Warehouse Database
Demonstration: Creating and Configuring an Azure SQL Data Warehouse Database
Question
Which of the following statements about a logical server is correct?
( )Option 1: You must create a new logical server each time you create a new database.
( )Option 2: A logical server is not a physical server.
( )Option 3: You are charged for each logical server you create.
( )Option 4: The more databases that a logical server hosts, the more the performance of the logical server will decrease.
( )Option 5: You can change the DWU performance of the logical server.
Answer
(√) Option -2: A logical server is not a physical server.

10. Create database in same process
Creating a Server
5: Implementing an Azure SQL Data Warehouse
Logical server
Specify:
Server name that has not been used
Server admin logon
Password
Location nearest to you
Create database in same process

11. Creating a Database Create database DWU settings Name of database
5: Implementing an Azure SQL Data Warehouse
Create database
Name of database
Drag slider to change DWU performance
Create a new server or use existing server
Source
Create a new resource group or use existing resource group
DWU settings
Scale
Pause/start

12. Configuring the Server Firewall
5: Implementing an Azure SQL Data Warehouse
Add client IP address before connecting
Client IP address may change
Specify rule:
Range of IP addresses to allow for change
IP addresses for other client computers
Mention that further information on how to manually specify a rule and add other IP addresses can be found at the forward link referenced in the manual.

13. Connecting to Azure Database Using SQL Server Management Studio
5: Implementing an Azure SQL Data Warehouse
Fully qualified server name
Connect to server using SSMS
USE statement not supported
Right mouse click database, New Query
Most Transact-SQL supported in Azure SQL Data Warehouse databases

14. In this demonstration, you will see how to:
Demonstration: Creating and Configuring an Azure SQL Data Warehouse Database
5: Implementing an Azure SQL Data Warehouse
In this demonstration, you will see how to:
Create an Azure SQL Data Warehouse Database and Server
Change the Performance Settings
Configure the Azure Firewall
Connect to the Azure Server with SQL Server Management Studio
Preparation Steps
Start the MSL-TMG1, 20767A-MIA-DC and 20767A-MIA-SQL virtual machines.
Demonstration Steps
Create an Azure SQL Data Warehouse Database and Server
Ensure that the MSL-TMG1, 20767A-MIA-DC and 20767A-MIA-SQL virtual machines are both running, and then log on to 20767A-MIA-SQL as ADVENTUREWORKS\Student with the password Pa$$w0rd.
In the D:\Demofiles\Mod05 folder, right-click Setup.cmd, and then click Run as administrator.
Click Yes when prompted to confirm that you want to run the command file, and then wait for the script to finish.
Open Microsoft Internet Explorer® and go to
Sign in to the Azure portal with your Azure pass or Microsoft account credentials.
Click New, click Data + Storage, and then click SQL Data Warehouse.
On the SQL Data Warehouse blade, in the Name box, type BikeSalesDW.
Drag the Performance slider to 200 DWU.
Under Server click Configure required settings, and then click Create a new server.
On the New server blade, in the Server name box, type 20767a followed by your initials, followed by today’s date. For example, if your initials are CN and the date is 15 March 2016, type acn15mar16. The server name must be unique; if the name is unique and valid, a green tick appears.
In the Server admin login box, type BikeSalesadmin.
In the Password and Confirm password boxes, type Pa$$w0rd.
Under Location, select a region nearest your current geographical location, and then click OK.
(More notes on the next slide)

15. Lesson 3: Developing an Azure SQL Data Warehouse
5: Implementing an Azure SQL Data Warehouse
User Defined Schemas
Question
A gender column in a table would be a suitable hashed column. True or false?
( )False
( )True
Answer
(√)False

16. Concurrency and Memory Allocation
5: Implementing an Azure SQL Data Warehouse
Resource Class
Concurrency Slots
Query may use more than one concurrency slot
Dependent on resource class and DWU service level
Concurrent Queries
Maximum of 32 queries
Maximum slots dependent on DWU service level
Memory Allocation

17. Data in Tables Allocated to Distributions Round-Robin Distribution
Data Distribution
5: Implementing an Azure SQL Data Warehouse
Data in Tables Allocated to Distributions
Round-Robin Distribution
Random distribution allocation
Hash Distribution
Choose hashed column
Distribution determined by function of column value
Ensure hashed column has even spread of data

18. Can set index properties and distribution type
CREATE TABLE AS SELECT
5: Implementing an Azure SQL Data Warehouse
Makes copy of a table
Can set index properties and distribution type
Use to work around unsupported features
CREATE TABLE Countries_New
WITH (
CLUSTERED COLUMNSTORE INDEX,
DISTRIBUTION = HASH(CountryKey) )
AS SELECT * FROM Countries ;

19. GROUP BY clause is supported
GROUP BY Limitations
5: Implementing an Azure SQL Data Warehouse
GROUP BY clause is supported
GROUPING SETS, CUBE and ROLLUP sub-clauses are not supported
UNION ALL operator is supported
When migrating to Azure SQL Data Warehouse, ensure queries containing unsupported clauses are amended

20. Local temporary tables can be accessed anywhere within session
5: Implementing an Azure SQL Data Warehouse
Local temporary tables can be accessed anywhere within session
Global temporary tables are not supported

21. All data in one database Use schemas to identify legacy databases
User Defined Schemas
5: Implementing an Azure SQL Data Warehouse
All data in one database
Use schemas to identify legacy databases
Mention that, if the legacy databases use schemas that you wish to maintain a reference to, you can prefix the name of each new table with a reference to the schema where the legacy table is situated.

22. Lesson 4: Migrating to an Azure SQL Data Warehouse
5: Implementing an Azure SQL Data Warehouse
Demonstration: Migrating a Database to Azure SQL Data Warehouse
Question
If you are migrating a schema using the Data Warehouse Migration Utility, and some of the data in the schema has features that are incompatible with Azure SQL Data Warehouse, the migration will continue.
( )False
( )True
Answer
(√)True

23. Using the Data Warehouse Migration Utility
5: Implementing an Azure SQL Data Warehouse
Advantages
Straightforward
Multiple tables
Specify distribution type
Notification of incompatibility
Download from Internet
Must have BCP and Excel installed

24. Differences Between SQL Server and Azure SQL Data Warehouse Schemas
5: Implementing an Azure SQL Data Warehouse
Some table features not supported
Primary Keys
Foreign Keys
Unique Indexes
Constraints
Some data types not supported
numeric
nvarchar(max)
varchar(max)
Mention the forward link in the manual for a more detailed list of features not supported.

25. Migrating Data Check compatibility Migrate schema Migrate data
5: Implementing an Azure SQL Data Warehouse
Check compatibility
Migrate schema
Migrate data
bcp commands to export and import

26. Updating Transact-SQL
5: Implementing an Azure SQL Data Warehouse
Some Transact-SQL not supported
Rewrite to achieve same result
Mention the forward link in the manual for the list of Transact-SQL not supported.

27. Demonstration: Migrating a Database to Azure SQL Data Warehouse
5: Implementing an Azure SQL Data Warehouse
In this demonstration, you will see how to:
Install the Data Warehouse Migration Utility
Check compatibility of the legacy database
Migrate the schema
Migrate the data
Demonstration Steps
Install the Data Warehouse Migration Utility
In the D:\Demofiles\Mod05\Installdwmigrationutility folder, double-click Data WarehouseMigrationUtility.msi.
If the Open File – Security Warning dialog box appears, click Run.
In the Data Warehouse Migration Utility dialog box, on the Select Installation Folder page, click Next.
On the License Agreement page, review the License Agreement, click I Agree, and then click Next.
In the User Account Control dialog box, click Yes.
On the Installation Complete page, click Close.
Minimize all open applications and note the Data Warehouse Migration Utility icon on the desktop.
Check Compatibility of the Legacy Database
In SQL Server Management Studio, in Object Explorer, under MIA-SQL, expand Databases, expand BikeSalesDW, and then expand Tables.
Right-click each of the following tables, click Select Top 1000 Rows, and then review the data they contain:
DimCustomers
DimProducts
DimResellers
FactInternetSales
FactResellerSales
(More notes on the next slide)

28. Lesson 5: Copying Data with the Azure Data Factory
5: Implementing an Azure SQL Data Warehouse
Data Factory Diagram
Question
In this lesson, the pipeline activity that copies data from the on-premises database to the Azure SQL Data Warehouse database does not contain any connection strings for the databases. Why is it not necessary to specify the connection strings?
Answer
The activity receives the connection strings indirectly by referring to the source and destination datasets. Each dataset references a linked server, which contains a connection string.

29. Capabilities and application to Azure SQL Data Warehouse Entities
The Azure Data Factory
5: Implementing an Azure SQL Data Warehouse
Capabilities and application to Azure SQL Data Warehouse
Entities
Activity
Pipeline
Dataset
Linked Service
Scheduling
JSON Templates
Edit parameters in script
Replace \ with \\

30. Creating a Data Factory
5: Implementing an Azure SQL Data Warehouse
Factory contains entities for activities
Specify
Name
Resource group name
Region

31. Setting Up a Data Gateway for the On-Premises Server
5: Implementing an Azure SQL Data Warehouse
Access data factory from on-premises server
Create new gateway
Install on computer

32. Setting up a Linked Service
5: Implementing an Azure SQL Data Warehouse
New data store
Edit parameters in JSON script
name
connectionString
Integrated Security
User ID
Password
gatewayName
userName: Use for Windows authentication
password: Use for Windows authentication

33. Edit parameters in JSON script
Setting up a Dataset
5: Implementing an Azure SQL Data Warehouse
New dataset
Edit parameters in JSON script
name
linkedServiceName
tableName
frequency
interval

34. Setting up a Pipeline Activity to Copy Data
5: Implementing an Azure SQL Data Warehouse
New pipeline
Edit parameters in JSON script
name
start
end
Add activity script

35. Data Factory Diagram Shows data flow Pipeline properties
5: Implementing an Azure SQL Data Warehouse
Shows data flow
Pipeline properties
Activities
Datasets
Dataset properties
Data slices

36. Lab: Implement an Azure SQL Data Warehouse
5: Implementing an Azure SQL Data Warehouse
Exercise 3: Copy Data with the Azure Data Factory
Point out that the lab instructions are deliberately designed to be high level so that students need to think carefully about what they are trying to accomplish, and work out how best to proceed. Encourage students to read the scenario information carefully and collaborate with each other to meet the requirements. Remind students that, if they find a particular task or exercise too challenging, there are step-by-step instructions in the lab answer key.
Exercise 1: Create an Azure SQL Data Warehouse Database
So that you can migrate your FoodOrdersDW database to Azure SQL Data Warehouse, you must first create a new Azure SQL Data Warehouse database using the Azure portal. As part of the process, you must also create a logical server for the database and configure the firewall settings so that you can connect to the server using SSMS on your computer. You will decide the name of the logical server, which must be globally unique.
Instructor Note: If students try to create their new database in task 2 on an existing server, they will be unable to specify a new resource group as described in the steps. In this scenario, advise students to create a new server rather than using an existing one. To make the server name unique, type the same server name as used previously, followed by lab.
Exercise 2: Migrate to an Azure SQL Data Warehouse Database
Now that you have implemented the FoodOrdersDW database on Azure SQL Data Warehouse, you are ready to migrate the data in the FoodOrdersDW database on the MIA-SQL server to the Azure SQL Data Warehouse database by using the Data Warehouse Migration Utility. You must first check the compatibility of the legacy data with Azure SQL Data Warehouse, migrate the schema, and then migrate the data in the tables.
Exercise 3: Copy Data with the Azure Data Factory
In the legacy data warehouse, the latest data for the call centre orders is currently entered into a table in a separate database called RecentCallCentreOrders. You are considering a scenario where you have a process that regularly copies the data in this table to a table in Azure SQL Data Warehouse. As a prototype, you want to schedule a pipeline activity in the Azure Data Factory that will copy the data in the CallCentreOrders table, in the RecentCallCentreOrders database on the MIA-SQL server, to the FactCallCentreOrders table in the Azure SQL Data Warehouse database.
Logon Information
Virtual machine: 20767A-MIA-SQL
User name: ADVENTUREWORKS\Student
Password: Pa$$w0rd
Estimated Time: 60 minutes

37. 20767A
Lab Scenario
5: Implementing an Azure SQL Data Warehouse
A data warehouse containing food orders might need to rapidly expand; however, this is not definite, so the executive board have decided not to purchase the hardware to support the expanded database and instead wish to implement an Azure SQL data warehouse. You have been asked to implement a preliminary test system that uses a cut down version of the data warehouse.

38. Lab Scenario (Continued)
5: Implementing an Azure SQL Data Warehouse
In this lab, you will create an Azure SQL Data Warehouse database on a new Azure logical server.
You will then use the Data Warehouse Migration Utility to migrate the data in the FoodOrdersDW database on the MIA-SQL server to the new Azure SQL Data Warehouse database. Finally, you will test the capabilities of the Azure Data Factory by setting up a scheduled pipeline activity that copies data from another database on the MIA-SQL server to a table in the new Azure SQL Data Warehouse database.

39. Module Review and Takeaways
5: Implementing an Azure SQL Data Warehouse
Review Question(s)
Review Question(s)
Question
How do you add other logins to an Azure SQL Data Warehouse logical server?
Answer
After you have created the logical server, you can connect to it using SQL Server Management Studio. You can then add the logons in the same way as you would with a SQL Server.