Demystifying the Cloud                                          Demystifying the Cloud What is this “Cloud” you speak of? Where does the Windows Azure Platform fit in? New Hampshire Code Camp 3 (Concord) 04-June-2011 Boston Azure User Group http://www.bostonazure.org @bostonazure Bill Wilder http://blog.codingoutloud.com @codingoutloud Copyright (c) 2011, Bill Wilder – Use allowed under Creative Commons license http://creativecommons.org/licenses/by-nc-sa/3.0/

Bill Wilder has been a software professional for over 20 years Bill Wilder has been a software professional for over 20 years. In 2009 he founded the Boston Azure User Group, an in-person cloud community which gets together monthly to learn about the Windows Azure platform through prepared talks and hands-on coding. Bill is a Windows Azure MVP, an active speaker, blogger (blog.codingoutloud.com), and tweeter (@codingoutloud) on technology matters and soft skills for technologists, a member of Boston West Toastmasters, and has a day job as a .NET-focused enterprise architect. Bill Wilder

Agenda What is the Cloud? Azure’s Place in Cloud Overview of Tables, Blobs, Queues, Drives Overview of Compute Resources Overview of Caching, Traffic Manager, ACS Time permitting… Scenarios for combining capabilities Azure Table Storage vs. Relational SQL Azure Roles & Queues

But first… DEMO

The Horse-Drawn Carriage From http://www.flickr.com/photos/historicbeaverton/2305428647/ Was shared under http://creativecommons.org/licenses/by-nd/2.0/ http://www.freefoto.com/preview/21-88-10/Vintage-Car Image may be licensed under Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License, FreeFoto FREE USElicense or see our Commercial Terms. http://www.freefoto.com/preview/29-35-19/Smart-Car

What Makes a Cloud a Cloud? METERED USAGE What are some ways a Cloud differs from a co-lo or ISP? PAY-AS-YOU-GO The cloud continues trend from differentiated hardware towards smarter software – the software-centric data center The cloud continues trend from differentiated hardware towards smarter software – the software-centric data center HOMOGENOUS, Over-PrOVISIONED COMMODITY HARDWARE “∞” Highly automatED VMs Generally speaking, a cloud is a pool of resources – like a smart phone, multi-tasking computer, or server farm – Think co-lo or ISP – these are the CHANGES, the next step Self-Service

“Bring Your Own” ____ as a Service BYO Users BYO Applications BYO Virtual Machines SaaS PaaS IaaS

___________________ as a Service Infrastructure Platform Software ___________________ as a Service Apps, $/user, LDAP, Expertise, SLA System Software OpEx, Auto Scale Out, Geo LB, Failover, HA, OS Patching, Monitoring, Monitoring, Backup, Expertise, SLA Hardware OpEx, Networking, DB/OS Licenses, Virtualization, Automation, Geo Distribution, CDN, Geo Replication, Elasticity, Managed Facility, Expertise, SLA BYO Users  Public Cloud Rental Models SaaS BYO Apps  PaaS DDTK = Dynamic Datacenter ToolKit … Dan will go into more detail on this in a few minutes Iceberg or Pyramid? BYO VMs  IaaS

“Bring Your Own” ____ as a Service BYO Users BYO Applications BYO Virtual Machines SaaS PaaS IaaS

Public  Hybrid  Private Public Cloud Platform My Data Center Public Cloud Hybrid Cloud Private Cloud

[Azure] Cloud Platform The rise of focused, specialized services Several “Scalable” Persistent Storage abstractions Illusion of infinite scale out Same performance with x clients as with 10x as with 100x as with 1000x as with … Billing model: pay only for what you use Friction-free access to add’l or less storage

Cloud-Native Applications Effort focuses on business functionality Development is highly productive Time-to-market is short Modification is straight-forward Infrastructure is not a limiting factor Cost structure is a good fit Downtime is not necessary Scale is efficient Innovation / experimentation is enabled

Azure Storage Services Blobs Drives Tables Queues SQL Azure

Persistent Storage Services – Options Type of Data Traditional Azure Way Relational SQL Server SQL Azure Blob File System, or SQL Server Azure Blobs File File System Azure Drives, or Azure Blobs Reliable Queue MSMQ (maybe) Azure Queues Non-Relational Azure Tables

Azure Storage Services Family of specialized, complementary storage services Abstractions High level services Blobs + Queues + Tables + Drives Many features in common Plus each with its own differentiating features SQL Azure

Azure Storage – Common Features RESTful API (http + XML + “resources”) http://bostonazure.table.core.windows.net/TableFoo?comp=list Operations - REST/http verb support Create = PUT Read = GET Update = POST Delete = DELETE Language agnostic Client library provided for .NET by Microsoft Many other client libraries available

Authentication Requirements PUT, POST, DELETE are always Authenticated GET is usually Authenticated Exception possible for Blobs Blob GET access can be public if Blob Container allows for Unauthenticated GET requests Sign http header using Access Key from Portal Don’t need to code this (use Microsoft-provided .NET client library) Key lives in an Azure xml config file

SQL Azure SQL Server in the cloud

SQL Azure is SQL Server Except… only SQL Azure only Full Text Search Size not limited Old school licensing model Encrypt, compress Many more… “Mirroring” Limitations Tables require a clustered index 50 GB size limit New Capabilities HA; Rental Coming: Backups & point-in-time recovery Other new Awesomeness… Common “Just change the connection string…” http://social.technet.microsoft.com/wiki/contents/articles/inside-sql-azure.aspx “Another feature in development is the ability to take control of your backups. Currently, backups are performed in the data centers to protect your data against disk or system problems. However, there is no way currently to control your own backups to provide protection against logical errors and use a RESTORE operation to return to an earlier point in time when a backup was made. The new feature involves the ability to make your own backups of your SQL Azure databases to your own on-premises storage, and the ability to restore those backups either to an on-premises database or to a SQL Azure database. Eventually Microsoft plans to provide the ability to perform SQL Azure backups across data centers and also make log backups so that point-in-time recovery can be implemented.” http://social.technet.microsoft.com/wiki/contents/articles/inside-sql-azure.aspx Additional information on Differences: http://msdn.microsoft.com/en-us/library/ff394115.aspx

SQL Azure Automagically handles… Provisioning on demand Install, configure, optimize (azurized™ versions of) “Windows Server 2008 R2” and “SQL Server 2008” Health monitoring Patching High Availability (HA) Load Balancing (LB) Billing

Not Only SQL http://www.cloudave.com/695/nosql-is-not-sql-and-thats-a-problem/ http://notonlysql.com/

NoSQL Databases Faster, Cheaper Scales Out “Simpler” , CouchDB: JSON Document Stores Amazon Dynamo, Azure Tables: Key Value Stores Many others! Faster, Cheaper Scales Out “Simpler” (And often eventually consistent) http://en.wikipedia.org/wiki/NoSQL http://en.wikipedia.org/wiki/CAP_theorem http://en.wikipedia.org/wiki/NoSQL

Relational Data vs. [NoSQL] Azure Tables Approach SQL Azure Azure Tables Normalization Normalized Denormalized (Duplication) (No duplication) (Lots of duplication) Structure Schema Flexible Transactions Distributed Limited scope Responsibility DBA/Database Developer/Code Knobs Many Few Scale Up (or Sharding) Out Consistency ACID semantics Strongly consistent http://en.wikipedia.org/wiki/NoSQL

Azure Table Storage – NoSQL in Azure Best place for granular, semi-structured data No rigid database schema No complex joins or complex transaction Fast and easy to instantiate Strongly Consistent No performance lag Programming model is WCF Data Services All data access and data updates LINQ Not SQL, not managed with SQL Server tooling

Azure Table Storage  DB Instance  Table  Row  Column Storage Account  DB Instance Table [*]  Table Entity [*, 1 MB data] PartitionKey + RowKey  Row Property [255x] Name/Value/Type Up to 64k  Column

Azure Table Storage Partition Key Row Key A “logical grouping” (a shard with less baggage) PartitionKey value of up to 64 KB Row Key Identify specific row within a partition RowKey value is String of up to 64 KB Table access requires Partition Key + Row Key Not to mention (cryptographic) Access Key (for the digital signing of the http header)

The File System for the Cloud Blob Container  Directory Blob  File Azure Blob Storage The File System for the Cloud Blob Container  Directory Blob  File http://msdn.microsoft.com/en-us/library/ff919703.aspx – Windows Azure CDN http://blogs.msdn.com/b/windowsazure/archive/2010/08/09/20-nodes-available-globally-for-the-windows-azure-cdn.aspx – 20 CDN nodes available across the world

Azure Blob Storage The place to put ANY LARGE OBJECT Private or Public Public blobs can be anonymously accessed Images, videos, CSS files Need not be binary Public blobs can be cached in Azure CDN 20 locations around the world Different locations than data centers Time-limited signed-access available “You have until tomorrow to download this video” http://msdn.microsoft.com/en-us/library/ff919703.aspx – Windows Azure CDN http://blogs.msdn.com/b/windowsazure/archive/2010/08/09/20-nodes-available-globally-for-the-windows-azure-cdn.aspx – 20 CDN nodes available across the world

Azure Blob Storage Two types of Azure Blobs “Block Blob” “Page Blob” Parallel upload scenarios Resume download scenarios Up to 200 GB “Page Blob” Random access scenarios Azure Drives Up to 1000 GB http://msdn.microsoft.com/en-us/library/ff919703.aspx – Windows Azure CDN http://blogs.msdn.com/b/windowsazure/archive/2010/08/09/20-nodes-available-globally-for-the-windows-azure-cdn.aspx – 20 CDN nodes available across the world

Azure Drive Storage Disk Drive simulation in cloud Façade over an Azure Blob Specifically, Page Blob type Legacy-focused, not “cloud native” Limits same as Page Blob limits 1000 GB = 1 TB

Azure Queue Storage Service for RELIABLE message delivery 7 days = default TTL for item to stay in queue 30 seconds = default “invisibility window” 8 KB = max size of a queued item 500 = approx number of transactions a queue can handle per second Beware of “spinning” – may get throttled, disabled N = number of queues you can have (N >> 1)

Replication http://commons.wikimedia.org/wiki/File:Stormtrooper_2.jpg

All Azure Storage is Replicated Resilient in case of failure “All data replicated multiple times” – Sriram Krishnan, Programming Windows Azure, p130+ Supports scale-out Replicate specific data if it gets too hot to keep up Geographic distribution of replication Currently up to you

http://www.freefoto.com/preview/04-28-6?ffid=04-28-6&k=Pile+of+Money

Pay As You Go – Data Transfer North America and Europe regions $0.10 per GB in $0.15 per GB out Asia Pacific Region $0.30 per GB in $0.45 per GB out No charge for transfer within a data center http://www.microsoft.com/windowsazure/pricing/

Pay As You Go Storage Windows Azure Storage SQL Azure $0.15 per GB stored per month $0.01 per 10,000 storage transactions “transactions” Queues, Tables, Blobs, Drives SQL Azure $10/GB/month http://www.microsoft.com/windowsazure/pricing/

Cost: SQL Azure=67x Table Storage $10/GB/m - SQL Azure $0.15/GB/m – Tables 10/0.15=66.66 No transaction costs Same data xfer 100x? Nominally, by SQL Azure max size SQL Azure not completely full Table Storage may denormalize

Pay As You Go – CDN Content Delivery Network (CDN) $0.15 per GB for data transfers from European and North American locations $0.20 per GB for data transfers from other locations $0.01 per 10,000 transactions More or less expensive than straight-to-blob costs Do the math! But also consider UX http://www.microsoft.com/windowsazure/pricing/

Azure Does Compute Compute Services Web Roles Worker Roles VM Role (coming?) Your Code Runs Here Image: http://t3.gstatic.com/images?q=tbn:lBdXOeap8CbUoM:http://regulus2.azstarnet.com/blogs/images/7064.jpg&t=1

Web Roles vs. Worker Roles Runs in IIS 7 (always listening) Built using ASP.NET, MVC Good to handle interactive users Addressable over Internet Good for hosting Web API (WCF) Runs Continuously Smarx “The only difference between a web role and a worker role is that in the web role, IIS (actually Hosted Web Core) gets started and pointed at your app data directory. You can still put code in WebRole.cs that does exactly the same thing as you would do in your worker role, so the decision should really be "Do you want IIS?" If so, use a web role. If not, use a worker role.” One is always running, one is always listening

“Out” is the New “Up” Scaling Out has hard limits at CPU, Memory Architecturally more limiting

Azure Storage Services in Concert http://commons.wikimedia.org/wiki/File:Boston_Pops_Esplanade_Orchestra_2005-07-04.jpg

AppFabric Services Not to be confused with the Fabric Controller Not to be confused with AppFabric for Windows Server Traffic Manager Caching (“Velocity”) Service Bus Connect Access Control Service (ACS)

Key Pattern: Roles + Queues Web Role (IIS) Worker Role Queues AJAX – orthogonal concern Worker Role not related to HTML 5 concept of Web Worker Blobs Tables

Canonical Example: Thumbnails Web Role (IIS) Worker Role Queues AJAX – orthogonal concern Worker Role not related to HTML 5 concept of Web Worker Blobs Tables

Roles + Queues: API queue.AddMessage( new CloudQueueMessage( statusUpdateMessage)); Web Role (IIS) Worker Role Queues AJAX – orthogonal concern Worker Role not related to HTML 5 concept of Web Worker CloudQueueMessage statusUpdateMessage = queue.GetMessage( TimeSpan.FromSeconds(10)); … queue.DeleteMessage(statusUpdateMessage);

Adding to Queue - Conceptual Queue Name: “thumbnailer-7” http://bostonazure.queue.core.windows.net/thumbnailer-7

Adding to Queue - Actual 314159 265358 979323 Azure Blob Storage 3.14159 265358 979323 846264 338327 950288 419716 939937 510582 097494 459230 314159 265358 979323

Geography From NASA: http://svs.gsfc.nasa.gov/vis/a000000/a002600/a002680/apollo17_earth_web.jpg Apollo 17

Geography Matters Select Data Center near customers Keep data resources near compute resources Currently 6 data centers from which to choose Additional 24 sites for CDN

Geography Matters Azure data centers Data Center choices Azure data centers North Central US South Central US West Europe North Europe East Asia Southeast Asia Anywhere options for: US, Europe, Asia

Slides available from Bill’s blog http://blog.codingoutloud.com hmbl.me is URL shortener running on Windows Azure: http://hmbl.me/6OD38B http://blog.codingoutloud.com/2010/07/14/key-architectural-design-pattern-for-cloud-native-azure-apps

Questions? Comments? More information?

Visit http://bit.ly/BillOnAzure Use this promo code: BillOnAzure Free 30-Day Windows Azure & SQL Azure Pass (in countries where Azure is offered) Visit http://bit.ly/BillOnAzure Use this promo code: BillOnAzure You will be provisioned an Azure account valid for 30 days that includes for FREE: Three small compute instances Two 1 GB SQL Azure Databases 3 GB of Windows Azure Storage And more…

Azure Pass Screen Shot

BostonAzure.org Boston Azure cloud user group Focused on Microsoft’s cloud platform Last Thursday, monthly, 6:00-8:30 PM at NERD Food; wifi; free; great topics; growing community Follow on Twitter: @bostonazure More info or to join our email list: http://www.bostonazure.org

I may be able to speak at your technology event Contact Me I may be able to speak at your technology event Just Ask! Bill Wilder @codingoutloud http://blog.codingoutloud.com