1. Windows Azure Conference 2014 Windows Azure SQL Database - Migration and Optimization

2. Windows Azure Conference 2014 Agenda Windows Azure SQL Database Migration Considerations Migration Tools and Options Performance Considerations Monitoring and Troubleshooting WA SQL DB Premium Database CSF Telemetry

3. Windows Azure Conference 2014 Windows Azure SQL Database Extremely fast & easy to provision – Seconds / Minutes. Flexibility & Built-in Manageability: Tools to manage SQL Azure: – SQL Server Management Studio, Azure Management Portal – PowerShell, Rest API Removes needs for patching. Built-in High Availability: Database replicated across 2 Instances across 2 data centres by default 99.9% Availability SLA Predictable Performance: Reserved resources through “Premium Offering” Elasticity: Quick Scale-out to thousands of distributed databases.

4. Windows Azure Conference 2014

5. Common Compatibility Considerations Features – Clustered Index is required – Non-default Collation should be at database level Some TSQL features not supported: Eg: XML Schema Collections Maximum 149 databases + master DB No cross-database joins Authentication using a user name and password Some Services don’t exist – SQL Agent – Distributed Transactions – Replication – Linked Server – Log Shipping Throttling – Resource consumptions, concurrent connections, idle connections etc

6. Windows Azure Conference 2014 Migration Migration involves transfer – Schema – Data Multiple tools to enable migration. Choosing tool for migration depends on – Type (SQL / Non SQL / Third Party) – Size – Complexity

7. Windows Azure Conference 2014 Data Transfer Tools

8. Windows Azure Conference 2014 SQL Server Migration Assistant Automate and simplify database migration Supports migration to: – SQL Server and SQL Azure Supports migration from: – Oracle, Sybase, MySQL, Access Migrates: – Schema – Data Using SSMA with WA SQL DB

9. Windows Azure Conference 2014 Convert the application, test, integrate, and deploy Assess the migration project Migration Analyzer Migrate the schema and business logic Schema Converter Migrate the data Data Migrator Test the converted database Migration Tester SQL Server Migration Assistant Available for MySQL, Oracle, Sybase, and Microsoft Access migrations Simplifying the Migration Process

10. Windows Azure Conference 2014 WA SQL Database Copy db1 Master Database CREATE DATABASE db1copy AS COPY OF db1 SELECT * FROM sys.dm_database_copies SELECT * FROM sys.databases db1 copy db1 copy db1copy Copies in Progress db1 - ONLINE db1copy - ONLINE

11. Windows Azure Conference 2014 DEMO Using Migration Tools to migrate WA SQL DB

12. Windows Azure Conference 2014

13. Effect of Connectivity Performance affected by connectivity – Multiple hops – Concurrent Connection Limit – DOS attacks – DB Failover

14. Windows Azure Conference 2014 Connectivity – Countermeasures Minimize distance from DB: To customer and application Open connections late, close early, Connection pooling Less Chatty communication: SPs, TVPs, batching Caching: Local, Azure Cache, CDN Prefer SQL Server Managed Transactions Consider ADO.NET async methods Use Transient Fault Handling Application Block

15. Windows Azure Conference 2014 Other DAL best practices ODBC / JDBC – Reduce cursor-based operations – Leverage read only, forward only and direct database interactions ADO.NET – Chose appropriate execution mode with SqlCommand – Batch multiple commands with DataAdapter or implement direct database interactions through SPs – Use binary remoting format for Dataset serialization Entity Framework – Avoid state tracking in object generation if not needed, avoid lazy loading – Use Stored Procedures for data store interactions – If not possible, at least use compiled queries and pre-generated views – Map multiple results set to object sets – Batch multiple insert/update/delete operations – Integrate (not specific support) with TVP and SqlBulkCopy

16. Windows Azure Conference 2014 Effect of Shared Resources and Throttling Resource shared with neighbors – CPU, memory, IO, Network – Temp DB, worker threads, SQL Engine Engine Throttling – Occurs when there is excessive resource usage – Blocks connectivity for read and/or write Soft : Affects subset of DBs Hard : Affects all DBs Resource Governance – Sets and monitors limit on worker threads (180) – Sets and monitors limit on sessions (internal) Resource Limits – DB Size (150 GB): – Txn. Duration (24 hrs), Total no of locks/txn (1M): – Temp DB (5 GB of Temp DB space, 2GB/txn): – Log space (2GB/txn), Memory Usage (16MB for > 20 sec): Throttling Engine Soft Hard Resource Governance Resource Limit

17. Windows Azure Conference 2014 Resource Bottlenecks - Countermeasures Transactions and Lock – Short Lived Transactions – Coarser grain lock hints – Monitor with following DMVs sys.dm_tran_active_transactions sys.dm_tran_database_transactions sys.dm_tran_locks sys.dm_tran_session_transactions DB Size Limits – Archival, Drop Unused Objects Temp DB, Txn Log length Drop temporary objects soon after usage Reduce number of rows in Txn scope Multiple short transactions Sharding Formula number of rows affected in table * (avg size of field being updated in bytes + 80) < 2 GB Memory Usage Query Tuning on queries with group by, order by etc

18. Windows Azure Conference 2014 Premium Database Dedicates a fixed amount of reserved capacity – For secondary replicas too Scenario Fitment – High resource consumption (CPU, Memory, IO) – Exceeds WA SQL DB concurrency limit (180) – Higher response time consistency (low latency) Current Sizes Available Reservation SizeCPU cores Concurrent active requests (workers) SessionsData IO (IOPS)Memory (GB) P1120020001508 P22400400030016

19. Windows Azure Conference 2014

20. Monitoring Use SQL Server System Views for Monitoring – Monitor top queries, deadlocks, resource consumption, sessions – Many sample queries are available Implement periodic monitoring policy Move monitored data to dedicated DB Ensure Monitoring itself is non-intrusive Implement analysis/reporting on data gathered Consider CSF Telemetry approach Tools – Cerebrata, NewRelic and others

21. Windows Azure Conference 2014 Useful System Views DMVScenario sys.dm_exec_query_memory_grants Queries waiting for memory before they can be executed. sys.dm_exec_cached_plans Execution plans that are currently in the case. sys.dm_db_missing_index_details Missing indexes that would increase the query performance. sys.dm_db_missing_index_columns Missing table columns for a given index. sys.dm_db_missing_index_groups Missing indexes are contained in a specific missing index group, excluding spatial indexes. sys.dm_db_missing_index_group_stats Groups of missing indexes, excluding spatial indexes. sys.dm_db_index_usage_stats Information about the usage of an index. sys.dm_db_index_physical_stats Information about the physical layout for a given index (space consumption etc). sys.dm_db_index_operational_stats Information about the performance for a given index. sys.dm_exec_procedure_stats Usage of stored procedures in the database. sys.dm_exec_trigger_stats Usage of triggers in the database.

22. Windows Azure Conference 2014 Premium Database and sys.resource_stats sys.resource_stats – Provides data about CPU, Memory, IOPS, Sessions for the database – One row for every five-minute reporting window There must have been a change in resource consumption (Idle databases might return no rows in the interval) – Historical data is retained for 14 days

23. Windows Azure Conference 2014 Demo Monitoring WA SQL DB using System Views

24. Windows Azure Conference 2014 CSF Telemetry Cloud Service Fundamentals Project – DAL with retry logic – Logging over NLOG – Integration with WAD – Data collection pipeline – Query-able Telemetry System – Sample Reports Works at high scale, with sharding Incorporates best practices for telemetry – Dedicated diagnostics account – Diag. data moves to table then to RDMBS – Std. log libraries, runtime config. Changes Free downloadable code project

25. Windows Azure Conference 2014