1. Bill Wilder – brief bio Bill Wilder has been a professional software developer for more than 20 years. Last year he founded the Boston Azure User Group, an in-person cloud computing community which gets together monthly to learn about Windows Azure through prepared talks and hands-on coding. Bill is especially excited about the Boston Azure Project, a collaborative Windows Azure coding project just starting up in the Boston Azure community. Bill is an active community speaker, blogger (blog.codingoutloud.com), and tweeter (@codingoutloud) on technology matters and soft skills for technologists, and is also a member of Boston West Toastmasters. Separately, Bill has a day job as an enterprise architect focusing on.NET.Boston Azure User GroupBoston Azure Projectblog.codingoutloud.com@codingoutloud

2. Azure 101 Boston Azure User Group 23-September-2010 Copyright (c) 2010, Bill Wilder Boston Azure User Group http://bostonazure.org @bostonazure Bill Wilder http://blog.codingoutloud.com http://blog.codingoutloud.com @codingoutloud Boston West Toastmasters http://bwtoastmasters.com http://bwtoastmasters.com Not here with my day job Only Bill’s personal views Windows Azure: A really quick look at Tables, Blobs, Queues, SQL Azure & Compute

3. Agenda for Azure 101 Azure’s Place in Cloud Overview of Tables, Blobs, Queues, Drives Overview of Compute Resources Scenarios for combining capabilities – Azure Table Storage vs. Relational SQL Azure What’s up with NoSQL? – Roles & Queues Cloud Scale

4. Azure Cloud Platform The rise of focused, specialized services Several “Scalable” Persistent Storage abstractions Illusion of infinite scale out Same performance with x clients as with 10x as with 100x as with 1000x as with … Billing model: pay only for what you use – Friction-free access to add’l or less storage

5. Cloud-Native Applications Effort focuses on business functionality – Development is highly productive – Time-to-market is short – Modification is straight-forward Infrastructure is not a limiting factor – Cost structure is a good fit – Downtime is not necessary – Scale is efficient Innovation / experimentation is enabled

6. ___________________ as a Service App Logic, Expertise Software CapEx, Load Balancing, Scale, Availability, Failover, Backup, OS Patching, Expertise, Programming Model, … Hardware CapEx (Servers, Routers), Virtualization, Networking, Geo Distribution, CDN, Large IT Staff, Facility Management, API, Expertise, …

7. Persistent Storage Services – Options Type of DataTraditionalAzure Way RelationalSQL ServerSQL Azure BlobFile System, or SQL Server Azure Blobs FileFile SystemAzure Drives, or Azure Blobs Reliable QueueMSMQ (maybe)Azure Queues Non-RelationalAzure Tables

8. Azure Storage Services Family of specialized, complementary storage services – Abstractions – High level services Blobs + Queues + Tables + Drives 1.Many features in common 2.Plus each with its own differentiating features

9. Azure Storage – Common Features RESTful API (http + XML + “resources”) http://bostonazure.table.core.windows.net/TableFoo?comp=list Operations - REST/http verb support Create = PUT Read = GET Update = POST Delete = DELETE Language agnostic Client library provided for.NET by Microsoft – Many other client libraries available

10. Authentication Requirements PUT, POST, DELETE are always Authenticated GET is usually Authenticated – Exception possible for Blobs – Blob GET access can be public if Blob Container allows for Unauthenticated GET requests Sign http header using Access Key from Portal – Don’t need to code this (use Microsoft-provided.NET client library) – Key lives in an Azure xml config file

11. Developer Portal for Keys https://windows.azure.com/

12. Storage Keys for Authentication

13. Azure Storage – Common Features Tables, Blobs, Queues, Drives all share the following features / characteristics Are all created under your Azure Storage Account Storage Account is usually created through the Developer Portal – Or API

14. Azure Storage – Common Features Support for metadata (8k, unstructured) Replicated so there are (at least) 3 copies – Within in a single data center – Strongly consistent programming model Easy, fast to dynamically add new containers – No limit to number of instances of blobs, tables, table rows, or queues Pay As You Go billing model

15. Azure Blob Storage The File System for the Cloud Blob Container  Directory Blob  File

16. Azure Blob Storage The place to put ANY LARGE OBJECT Private or Public Public blobs can be anonymously accessed – Images, videos, CSS files – Need not be binary Public blobs can be cached in Azure CDN – 20 locations around the world – Different locations than data centers Time-limited signed-access available – “You have until tomorrow to download this video”

17. Azure Blob Storage Two types of Azure Blobs 1.“Block Blob” – Parallel upload scenarios – Resume download scenarios – Up to 200 GB 2.“Page Blob” – Random access scenarios – Azure Drives – Up to 1000 GB

18. Azure Drive Storage Disk Drive simulation in cloud Façade over an Azure Blob – Specifically, Page Blob type Legacy-focused, not “cloud native” Limits same as Page Blob limits – 1000 GB = 1 TB

19. Azure Table Storage Best place for granular, semi-structured data – No rigid database schema Fast and easy to instantiate – Strongly Consistent – No performance lag Programming model is WCF Data Services – All data access and data updates – LINQ

20. Azure Table Storage Storage Account Table [*] Entity [*, 1 MB data] PartitionKey + RowKey Property [255x] Name/Value/Type Up to 64k

21. Azure Table Storage Partition Key – Along a “logical grouping” – a “shard” – PartitionKey value of up to 64 KB Row Key – Identify specific row within a partition – RowKey value is String of up to 64 KB Table access requires Partition Key + Row Key – Not to mention (cryptographic) Access Key (for the digital signing of the http header)

22. Relational Data vs. Azure Tables ApproachSQL AzureAzure Tables NormalizationNormalizedDenormalized (Duplication)(No duplication)(Lots of duplication) StructureSchemaFlexible TransactionsDistributedLimited scope ResponsibilityDatabaseDeveloper KnobsManyFew ScaleUp (or Sharding) Out CostReasonableLess expensive

23. Azure Queue Storage Service for RELIABLE message delivery 7 days = default TTL for item to stay in queue 30 seconds = default “invisibility window” 8 KB = max size of a queued item 500 = approx number of transactions a queue can handle per second – Beware of “spinning” – may get throttled, disabled N = number of queues you can have (N >> 1)

24. Replication

25. All Azure Storage is Replicated Resilient in case of failure – “All data replicated multiple times” – Sriram Krishnan, Programming Windows Azure, p130+ Supports scale-out – Replicate specific data if it gets too hot to keep up Geographic distribution of replication – Currently up to you

27. Pay As You Go – Storage Storage – $0.15 per GB stored per month – $0.01 per 10,000 storage transactions Applies equally to: – Queues – Tables – Blobs – Drives

28. Pay As You Go – Data Transfer North America and Europe regions – $0.10 per GB in – $0.15 per GB out Asia Pacific Region – $0.30 per GB in – $0.45 per GB out No charge for transfer within a data center

29. Pay As You Go – CDN Content Delivery Network (CDN) – $0.15 per GB for data transfers from European and North American locations – $0.20 per GB for data transfers from other locations – $0.01 per 10,000 transactions Same or less expensive than straight-to-blob costs

30. Azure Does Compute Compute Services Web Roles Worker Roles VM Role (coming?) Your Code Runs Here

31. Web Roles vs. Worker Roles Web RoleWorker Role Runs in IIS 7 (always listening) Built using ASP.NET, MVC Good to handle interactive users Addressable over Internet Good for hosting Web API (WCF) Runs Continuously

32. “Out” is the New “Up” Scaling Out has hard limits at CPU, Memory – Architecturally more limiting

33. Azure Storage Services in Concert

34. Key Pattern: Roles + Queues Web Role (IIS) Web Role (IIS) Worker Role Worker Role Queues Blobs Tables

35. Canonical Example: Thumbnails Web Role (IIS) Web Role (IIS) Worker Role Worker Role Queues Blobs Tables

36. Roles + Queues: API Web Role (IIS) Web Role (IIS) Worker Role Worker Role Queues queue.AddMessage( new CloudQueueMessage( statusUpdateMessage)); CloudQueueMessage statusUpdateMessage = queue.GetMessage( TimeSpan.FromSeconds(10)); … queue.DeleteMessage( statusUpdateMessage );

37. Queue Name: “thumbnailer-7” http://bostonazure.queue.core.windows.net/thumbnailer-7 Adding to Queue - Conceptual

38. Azure Blob Storage Adding to Queue - Actual 314159265358979323

39. Geography

40. Geography Matters Select Data Center near customers Keep data resources near compute resources Currently 6 data centers from which to choose Additional 20 sites for CDN

41. Geography Matters Azure data centers 1.North Central US 2.South Central US 3.West Europe 4.North Europe 5.East Asia 6.Southeast Asia Anywhere options for: US, Europe, Asia Data Center choices

42. Slides available from Bill’s blog http://blog.codingoutloud.com hmbl.me is URL shortener running on Azure: http://hmbl.me/2FPW3L http://blog.codingoutloud.com/2010/07/14/key-architectural-design-pattern-for-cloud-native-azure-apps

43. BostonAzure.org Boston Azure cloud user group Focused on Microsoft’s cloud solution Next meeting: 6-8 PM Thurs Oct 21 nd 2010 – Hacking on “Boston Azure Project” – Ben Day on Azure + Silverlight: Better Together Meetings usually 4 th Thursday of month – No cost; food; great topics; growing community Join email list: http://bostonazure.org Follow on Twitter: @bostonazure

44. Bill Wilder @codingoutloud http://blog.codingoutloud.com