1. Windows Azure Data Services Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg

2. Agenda - Overview - Management Portal / Tools - Windows Azure Storage - Tables, Blobs, Queues - Windows Azure SQL Database - Putting them together with Cloud Design Pattern Examples

3. Windows Azure Storage Overview Management Portal Storage Emulator / Tools

4. What is Windows Azure Storage? - Storage as a service – Blobs, Tables, and Queues - Lots of it (up to 200 TB) – pay as you go - Highly Available - Redundancy - Highly Scalable - Partitions - Blazing Speed - Performance “Targets”

5. Getting Started with Storage - Create a storage account in the portal - Connection string Account Name Account Key

6. Getting Started with Storage - Use storage emulator for development - Connection string UseDevelopmentStorage=true

7. Windows Azure SDK Windows Azure SDK Visual Studio 2013 Visual Studio 2012 Visual Studio 2010 2.2 Oct2013Supported Not Supported 2.1 Jul2013Not SupportedSupported 2.0 Apr2013Not SupportedSupported - SDK contains client libraries - Committed to backwards compatibility

8. Storage Client Library - SCL 2.1 included with SDK 2.2 - SCL 3.0.x upgrade available with NuGet

9. Tools to interact with storage Visual Studio Server Explorer CloudXplorer/TableXplorer AzureXplorer (VS Extension)

10. Demo Management Portal Storage Tools

11. Windows Azure Storage Tables Blobs Queues

12. Tables Concept Overview AccountTableEntity gwabjax racers results Name=Ricky Status=Active Name=Dale Status=Active Race=Jax 500 Date=3/29/14

13. - Structured storage in the form of tables -1MB max size -252 properties - Quick queries by clustered index (only index) - 2,000 entities per second per partition - Data-defined partition scheme -Table name + PK Table Details

14. - Tables store data more like key-value pairs - No schema - De-normalized structure optimized for performance Table Details First NameLast NameDate of BirthLast Win RickyBobby1/1/1971 DaleEarnhardt2/2/1972Daytona 500 CalNaughtonMarch 3, 1973

15. Table Entities - Property Types byte[] bool DateTime double Guid Int32 Int64 String

16. Table Entities - Required properties PartitionKey, RowKey, Timestamp - Operations on entities Delete Insert Upsert (Insert + Merge/Replace) Update (Merge or Replace) Retrieve Query

17. Using Tables 1.Get table reference 2.Execute operations on entities CloudStorageAccount storageAccount = CloudStorageAccount.Parse( CloudConfigurationManager.GetSetting("DefaultConnection")); CloudTableClient tableClient = storageAccount.CreateCloudTableClient(); CloudTable table = tableClient.GetTableReference("racers"); TableOperation upsertOperation = TableOperation.InsertOrReplace(racer); table.Execute(upsertOperation);

18. Using Tables with DTOs/POCOs 1.Get table reference (like before) 2.Execute operations on entities via EntityAdapter With TableEntity – RacerEntity var operation = TableOperation.InsertOrReplace(racerEntity); table.Execute(operation); With DTO – Racer var adapter = new RacerAdapter(racer); var operation = TableOperation.InsertOrReplace(adapter); table.Execute(operation);

19. Table Best Practices 1.Avoid querying across partitions 2.Batch inserts (within partition) - Order and details in same table/partition 3.Storing aggregate copies - Ensures 1 query  1 partition - Format data up front 4.Use intelligent PKs

20. Demo Tables Blobs Queues

21. Blobs Concept Overview AccountContainerBlob gwabjax photos videos DSC000256.j pg teamlogo.png 1stPlace.avi http://.blob.core.windows.net/ /

22. Blobs - Types of Blobs - Page - Block - Operations on blobs Exists List Upload from Stream, ByteArray, Text, File Download to Stream, ByteArray, Text, File Delete

23. Using Blobs 1.Get blob container reference 2.Perform operations with blobs CloudStorageAccount account = CloudStorageAccount.Parse( CloudConfigurationManager.GetSetting("DefaultConnection")); CloudBlobClient client = account.CreateCloudBlobClient(); CloudBlobContainer container = client.GetContainerReference("photos"); CloudBlockBlob blockBlob = container.GetBlockBlobReference(refId); blockBlob.UploadFromStream(stream); blockBlob.Metadata["FileName"] = fileName; blockBlob.SetMetadata();

24. Demo Tables Blobs Queues

25. Queues Concept Overview AccountQueueMessages gwabjax orders emails-to- send Bulk http://... Small http://...

26. Queues - Operations on queues Add Peek Get Update Delete - Queue naming - All lowercase - Alpha-numeric and hyphen “-” - 3-63 characters

27. ClientWeb RoleStorage Queues in the wild Worker Role Browse r Web App Get Survey Complete Survey Post Results Surveys Queue Proces s Task Thank you Add Get Delete Process*

28. ClientWeb RoleStorage Queues in the wild Worker Role Blob Store Browse r Web App Get Survey Complete Survey Post Results Small Surveys Large Surveys Small Task Big Task Small Survey Get Delete Process* Add Thank you

29. ClientWeb RoleStorage Queues in the wild Worker Role Blob/Tabl e Store Browse r Web App Get Survey Complete Survey Post Results Small Surveys Queue Large Surveys Queue Small Task Large Task Large Survey Get Delete Process* Add Thank you Upload Download

30. Using Queues 1.Get queue reference 2.Perform operations with messages CloudStorageAccount account = CloudStorageAccount.Parse( CloudConfigurationManager.GetSetting("DefaultConnection")); CloudQueueClient client = account.CreateCloudQueueClient(); CloudQueue queue = client.GetQueueReference("orders"); CloudQueueMessage message = new CloudQueueMessage("ticket-order-17"); queue.AddMessage(message);

31. Demo Tables Blobs Queues

32. Windows Azure SQL Database (formerly SQL Azure) Management Portal Data Access

33. Windows Azure SQL Database Tools - SSMS - Visual Studio/SSDT - Modeling tools Code -ADO.NET -Enterprise Library -ORMs (e.g. Entity Framework, nHibernate)

34. Key Features -High Availability/Durability -Three Replicas (1 Primary – 2 Secondary) -99.9% SLA -Manageability -Portal, PowerShell, SSMS, REST API -Predictable Performance -Scale-out*

35. What’s Different – Maybe Bad? Missing -Extended stored procedures -SQL-CLR -Service Broker -Table partitioning Important: All tables must have a clustered index

36. Data Types? Data Type CategoryWindows Azure SQL Database Support Exact numericsbigint, bit, decimal, int, money, numeric, smallint, smallmoney, tinyint Approximate numericsfloat, real Date and timedate, datetime2, datetime, datetimeoffset, smalldatetime, time Character stringschar, varchar, text Unicode character strings nchar, nvarchar, ntext Binary stringsbinary, varbinary, image Spatial data typesgeography, geometry Other data typescursor, hierarchyid, sql_variant, table, timestamp, uniqueidentifier, xml

37. Development Story -Local SQL 2012/Express database -SSDT Database projects -Deploy directly to Azure from Visual Studio -Create deployment package -Use Release Management -Azure Connection String <add name="DefaultConnection" connectionString="Server=tcp:[server].database.windows.net,1433; Database=[database]; User ID=[database]-host@[server]; Password=[password]; Encrypt=true;" providerName="System.Data.SqlClient"/>

38. Demo Management Portal Create Server Create Database Firewall Rules

39. Patterns at Work Cache-aside Pattern Materialized View Pattern Competing Consumers Pattern Compensating Transaction Pattern

40. Cache-Aside Pattern -Build cache on-demand -Read cache first -If not there -Get from data store -Store in cache

41. Materialized View Pattern -Improve performance in systems with difficult queries -Pre-populated views -Not updated by app, can be entirely rebuilt -Inherent delays -Doesn’t query well

42. Competing Consumers Pattern -Protect against a large influx (burst) of requests -Balanced workload -Scalable

43. Compensating Transaction Pattern -Eventual consistency -Series of autonomous steps -Intermediate steps appear inconsistent -Compensating Transaction -Intelligent process to undo succeeded steps

44. Compensating Transaction Pattern -Example: Itinerary Creation -Book 3 flights and 2 hotel rooms -If one or more are unavailable, we start over

45. Compensating Transaction Pattern -Example: Itinerary Creation

46. What did we learn? -Azure data storage options/uses -Each has pros/cons/best fit -Management Portal and Tools -Design patterns

47. Questions? Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg

48. Thank you! Scott Klueppel Chief Cloud Evangelist SOAlutions, Inc. #gwab @kloopdogg

49. References Cloud Design Patterns (P&P) http://msdn.microsoft.com/en-us/library/dn568099.aspx http://msdn.microsoft.com/en-us/library/dn568099.aspx Data Services – Storage http://msdn.microsoft.com/en-us/library/windowsazure/gg433040.aspx http://msdn.microsoft.com/en-us/library/windowsazure/gg433040.aspx Storage Differences – Emulator vs Cloud http://msdn.microsoft.com/en-us/gg433135 http://msdn.microsoft.com/en-us/gg433135 Data Services – SQL Database http://msdn.microsoft.com/en-us/library/windowsazure/ee336279.aspx http://msdn.microsoft.com/en-us/library/windowsazure/ee336279.aspx